Image search method, device, computer device, and computer program

Abstract

To provide an image search method and apparatus, a computer device, and a storage medium, configured to implement image search using an image in an instant messaging client.SOLUTION: An image search method and apparatus, a computer device, and a storage medium are disclosed. The method includes the steps of: displaying an image selection page of an instant messaging client; determining, based on an operation performed by a user on the image selection page, an input image for a search; presetting a plurality of image similarity dimensions according to levels of similarity between two chat interactive-images; searching for at least one output image similar to the input image in the plurality of image similarity dimensions, each output image being similar to the input image in at least one image similarity dimension; and generating a search result list according to the at least one output image, and displaying the search result list.SELECTED DRAWING: Figure 1b

Description

[Technical field]

[0001] The present application relates to the technical field of the Internet, and more particularly to an image search method, device, computer device, and storage medium.

[0002] This application claims priority to a Chinese patent application filed with the China Patent Office on June 12, 2019, bearing application number 201910507945.5 and title "Image search method, apparatus, computer equipment and storage medium." [Background technology]

[0003] An IM (instant messaging) application is software that realizes online chat and communication by instant messaging technology, and users can exchange messages with other users through the chat application to realize chat. Currently, in order to improve the fun of chat, instant messaging applications can also support sending images, such as stamps, to interact with chat partners. However, the prior art does not have a scheme for searching images by images. Summary of the Invention [Problem to be solved by the invention]

[0004] The embodiments of the present application provide an image search method, apparatus, computer device and storage medium, which can realize image search by image in an instant messaging client. [Means for solving the problem]

[0005] An embodiment of the present application provides an image retrieval method, displaying an image selection page of the instant messaging client; determining an input image for a search based on an operation performed by a user on the image selection page; presetting a number of image similarity dimensions according to a similarity level between two chat dialogue images; searching for at least one output image similar to the input image in the plurality of image similarity dimensions, each output image being similar to the input image in at least one image similarity dimension; and Generating and displaying a search result list in response to the at least one output image.

[0006] Similarly, an embodiment of the present application provides yet another image retrieval method, determining an input image for the search based on an image search request sent from the terminal; presetting a number of image similarity dimensions according to a similarity level between two chat dialogue images; searching for at least one output image similar to the input image in the plurality of image similarity dimensions, each output image being similar to the input image in at least one image similarity dimension; generating a search result list in response to the at least one output image; and Returning the search result list to the terminal and causing the terminal to display it on an interface of an instant messaging client.

[0007] Similarly, an embodiment of the present application further provides an image retrieval device, including a first display unit, a determining unit, a setting unit, a retrieval unit, a generating unit, and a second display unit; the first display unit is used for displaying an image selection page of an instant messaging client; The determination unit is used for determining an input image for searching according to an operation performed by a user on the image selection page; The setting unit is used for presetting a plurality of image similarity dimensions according to a similarity level between two chat dialogue images; The searching unit is used for searching and obtaining at least one output image similar to the input image in the plurality of image similarity dimensions, each output image being similar to the input image in at least one image similarity dimension; the generating unit is adapted to generate a search result list according to the at least one output image; The second display unit is used for displaying the search result list.

[0008] Similarly, an embodiment of the present application further provides an image retrieval device, comprising: a determining unit, a setting unit, a retrieval unit, a generating unit, and a sending unit; The determining unit is used for determining an input image according to an image search request sent from a terminal; The setting unit is used for presetting a plurality of image similarity dimensions according to a similarity level between two chat dialogue images; The searching unit is used for searching and obtaining at least one output image similar to the input image in the plurality of image similarity dimensions, each output image being similar to the input image in at least one image similarity dimension; the generating unit is adapted to generate a search result list according to the at least one output image; The sending unit is used for returning the search result list to the terminal, so as to cause the terminal to display it on an interface of an instant messaging client.

[0009] An embodiment of the present application further provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and operable on the processor, the processor executing the program to implement steps of any of the image search methods provided by the embodiments of the present application.

[0010] In addition, the embodiments of the present application further provide a storage medium, on which a computer program is stored, and which, when executed by a processor, realizes the steps of any of the image retrieval methods provided by the embodiments of the present application.

[0011] In order to more clearly describe the technical solutions of the embodiments of the present application, the following briefly describes the drawings that need to be used in the description of the embodiments. It is obvious that the drawings in the following description are only some embodiments of the present application, and those skilled in the art can obtain other drawings based on these drawings without making creative efforts. [Brief description of the drawings]

[0012] [Figure 1a] FIG. 2 is a schematic diagram of a scene of an image search method provided by an embodiment of the present application; [Figure 1b] 1 is a flowchart of an image search method provided by an embodiment of the present application. [Figure 1c] FIG. 2 is a schematic diagram of a page operation of stamp search provided by an embodiment of the present application; [Figure 1d] FIG. 2 is a schematic diagram of a stamp search page display provided by an embodiment of the present application. [Figure 1e] FIG. 2 is a schematic diagram of a trigger display of a stamp search page provided by an embodiment of the present application. [Figure 1f] FIG. 2 is a schematic diagram of output stamp transmission provided by an embodiment of the present application; [Figure 2a] FIG. 13 is a schematic diagram of another page operation of output stamp search provided by an embodiment of the present application. [Figure 2b] FIG. 13 is a schematic diagram of another page operation of output stamp search provided by an embodiment of the present application. [Figure 2c] FIG. 13 is a schematic diagram of another page operation of output stamp search provided by an embodiment of the present application. [Figure 2d] FIG. 13 is a schematic diagram of another page operation of output stamp search provided by an embodiment of the present application. [Figure 2f] 4 is a flowchart of an image search method provided by another embodiment of the present application; [Figure 3a] 4 is a flowchart of an image search method provided in yet another embodiment of the present application. [Figure 3b] FIG. 2 is a schematic diagram of an architecture for output stamp search provided by an embodiment of the present application; [Figure 3c] FIG. 2 is a process diagram of feature extraction provided by an embodiment of the present application. [Figure 3d] FIG. 2 is a schematic diagram of an output stamp recall provided by an embodiment of the present application. [Figure 3e] FIG. 2 is a schematic diagram of output stamp sorting provided by an embodiment of the present application; [Figure 4a] FIG. 2 is a schematic diagram of an output stamp list provided by an embodiment of the present application. [Figure 4b] FIG. 13 is a schematic diagram of another output stamp list provided by an embodiment of the present application. [Figure 4c] FIG. 13 is a schematic diagram of another output stamp list provided by an embodiment of the present application. [Figure 5a] FIG. 1 is a structural schematic diagram of an image search device provided by an embodiment of the present application; [Figure 5b] FIG. 2 is another structural schematic diagram of an image search device provided by an embodiment of the present application; [Figure 6] FIG. 2 is another structural schematic diagram of an image search device provided by an embodiment of the present application; [Figure 7] FIG. 2 is a structural schematic diagram of a computer device provided by an embodiment of the present application; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] Hereinafter, the technical solutions in the embodiments of the present application will be described clearly and completely with reference to the drawings of the embodiments of the present application. It is obvious that the described embodiments are only some embodiments of the present application, not all embodiments. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative efforts are all within the scope of protection of the present application.

[0014] The embodiments of the present application provide an image search method, an apparatus, a computer device, and a storage medium. Specifically, the embodiments of the present application provide an image search apparatus applicable to a first computer device (which may be referred to as a first image search apparatus for distinction) and an image search apparatus applicable to a second computer device (which may be referred to as a second image search apparatus for distinction). The first computer device may be a device such as a terminal, and the terminal may be a device such as a mobile phone, a tablet PC, a notebook computer, etc. The second computer device may be a network side device such as a server.

[0015] For example, the first image search device may be integrated into a terminal, and the second image search device may be integrated into a server, which may be a single server or a server cluster consisting of multiple servers.

[0016] In the embodiment of the present application, the image search method is described by taking an example in which the first computing device is a terminal and the second computing device is a server.

[0017] As shown in FIG. 1a, an embodiment of the present application provides an image retrieval system 100, including a terminal 110 and a server 120, and the terminal 110 and the server 120 are connected by a network, such as a wired or wireless network. An image retrieval device 111 is integrated in the terminal 110, for example, in the form of a client. The terminal 110 displays an image selection page of an instant messaging client, determines an input image for retrieval according to an operation performed by a user on the image selection page, pre-sets a number of image similarity dimensions according to the similarity level between two chat dialogue images, and retrieves at least one output image similar to the input image according to the number of image similarity dimensions. Each output image may be similar to the input image in at least one image similarity dimension, and may be used to generate and display a search result list according to the at least one output image.

[0018] Wherein, the terminal 110 may have multiple ways of triggering the output image search, for example, the terminal 110 may trigger the server 120 to search for output images similar to the input image in multiple image similarity dimensions. Specifically, the terminal 110 may send an image search request to the server 120.

[0019] Specifically, the server 120 determines an input image for searching according to an image search request sent from a terminal, pre-sets a number of image similarity dimensions according to the similarity level between two chat dialogue images, and searches and obtains at least one output image similar to the input image in the number of image similarity dimensions, where each output image is similar to the input image in at least one image similarity dimension, generates a search result list according to the at least one output image, and returns the search result list to the terminal for displaying on an interface of an instant messaging client on the terminal.

[0020] The embodiment of the present application is described from the perspective of a first image search device, which may be specifically integrated into a terminal. The embodiment of the present application provides an image search method, which may be executed by a processor of a terminal 110 as shown in Fig. 1a, and the specific process of the image search method includes the following steps as shown in Fig. 1b:

[0021] 101: Display an image selection page of an instant messaging client.

[0022] The image in the embodiment of the present application may include a dialogue image applied to an instant messaging client, or a dialogue image used by a user in an instant messaging client, and may be, for example, a chat dialogue image. The chat dialogue image is an image of a dialogue with a chat partner in a chat scene, and may include, for example, an image expressing information to a chat partner in a chat conversation, such as an image expressing emotions and words. For example, the chat dialogue image may include stamps, etc. The image type of the chat dialogue image may be multiple, and may include, for example, a still image or a dynamic image. For example, the chat dialogue image may include a still stamp, a dynamic stamp, etc.

[0023] In an embodiment of the present application, before displaying the image selection page, an image search page may be displayed. The image search page may be a search page for a user to search images by images, and the user may operate on the image search page to trigger searching for output images by images (image search by image). For example, the image search page may be a page for a user to search chat dialogue images by chat dialogue images, and the user may perform an image search operation on the page, for example, the image search page may include a stamp search page, and the user may perform a search operation on the stamp search page to search for a required output stamp by stamp (i.e., stamp search by stamp), etc., as shown in FIG. 1c.

[0024] Wherein, the image search page may include an image selection control, which can be used by a user to trigger the display of an image selection page to determine an input image, wherein the representation form of the control may be an icon, an input box, a button, etc.

[0025] For example, referring to FIG. 1c, taking the image as a stamp, stamp search page 1c1 may include an image selection control, i.e., a "+" plus sign button, and when a user needs to select an input stamp for search, the user operates the plus sign button, for example, by clicking, sliding, etc., to trigger the display of stamp selection page 1c2. The stamp selection page 1c2 displays a stamp selection panel including a candidate stamp list, which includes stamps that the user has favorited, and the stamp selection panel allows the user to select or determine the input stamp.

[0026] In an embodiment of the present application, a user may perform an operation on a user operation page of an instant messaging client, thereby triggering the display of an image search page of the instant messaging client, such as a stamp search page 1c1. For example, in one embodiment, a user may perform an operation on a content search page of an instant messaging client, thereby triggering the display of an image search page. Among them, the content search page may be a page for a user to search for content, such as news, articles, application programs, such as mini programs, business services, images (such as chat dialogue images), etc., and the user may perform a search operation on the content search page to search for the required content.

[0027] For example, a chat dialogue image search control may be configured on a content search page such that a user can interact with the control to trigger display of an image search page, specifically including displaying a content search page of an instant messaging client, the content search page including an image search control, and if a user's triggering interaction with the image search control is detected, displaying an image search page of the client, and the image search page including an image selection control, and if a user's triggering interaction with the image selection control is detected, displaying the image selection page.

[0028] For example, taking the image as a chat sticker, as shown in FIG. 1d, the terminal may display a content search page of the instant messaging client, where the content search page may include multiple content search controls, such as a text search control, a mini program search control, a music search control, an official account search control, and a sticker search control. The content search page may also include hotspot content in the instant messaging client. When a user needs to search for a sticker, the user may perform an operation on the sticker search control in the content search page, such as clicking, sliding, etc., and when the terminal detects the user's operation on the sticker search control, it triggers the display of a sticker search page 1c1 shown in FIG. 1c.

[0029] In the embodiment of the present application, there may be multiple ways to trigger the display of the content search page, for example, a user may operate on a chat conversation list page to trigger the display of the content search page, for example, a content search control may be set on the chat conversation list page, and the user may trigger the display of the content search page by operating on the control. Specifically, the present application may include displaying a chat conversation list page of an instant messaging client, the chat conversation list page including a content search control, and displaying the content search page when a user's trigger operation on the content search control is detected.

[0030] Wherein, the chat conversation list page is a page for displaying a chat conversation list, and the chat conversation list may include one or more chat conversations (e.g., conversations of a single chat, a group chat, etc.). For example, as shown in Fig. 1e, when a terminal displays a chat conversation list page of an instant messaging client, for example, the left page in sub-figure 1e1, a global content search control is set on the page, and when a user operates the global content search control, for example, performs an operation such as clicking, sliding, etc., the terminal displays the right content search page in sub-figure 1e1.

[0031] Also, for example, the user may further perform an operation on a function page of the instant messaging client to trigger the display of a content search page, for example, a search function control may be set on the function page, and the user may perform an operation on the control to trigger the display of the content search page. Specifically, the method may include displaying a function page of the instant messaging client, the function page including a search function control, and displaying the content search page when a user's trigger operation on the search function control is detected.

[0032] For example, as shown in FIG. 1e, the terminal displays a chat conversation list page of an instant messaging client, for example, the page on the left in sub-figure 1e2, and a global content search control is set on the page. When the user operates on the global content search control, the terminal displays a function page on the right in sub-figure 1e2, for example, a “Discover” page, and the page includes function controls such as a QR code (registered trademark) scanning function control “Scan”, a content posting control “Moments”, a mini program function control “Mini Program”, and a search function control “Search”, and when the user operates on the search function control “Search”, for example, by clicking, sliding, etc., the terminal displays the content search page shown in FIG. 1d.

[0033] 102, determining an input image for search based on the operation performed by the user on the image selection page.

[0034] Among these, the input image determination operation performed on the image selection page may be a single operation, for example, the user performs a single click operation on the selection page, or it may be an operation consisting of multiple, for example, a series of operations, for example, the user performs multiple different operations on the selection page.

[0035] 103, preset a number of image similarity dimensions according to the similarity level between two chat dialogue images.

[0036] 104, searching for and obtaining at least one output image similar to the input image in a plurality of image similarity dimensions, where each output image is similar to the input image in at least one image similarity dimension;

[0037] 105, generating and displaying a search result list according to the at least one output image.

[0038] Among them, the display manner of the search result list may be related to the manner of determining the input image. Below, several search result display methods in different determination manners of the input image are described.

[0039] (1) The user selects an input image from a list of candidate images on an image selection page.

[0040] Wherein, the image selection page may include a candidate image list, which may include at least one candidate image, for example, when the image is a stamp, the candidate image list may be a candidate stamp list, which may include at least one candidate stamp. For example, the candidate image list may include at least one of stamps including stamps favorited by the user in the client, stamps unique to the client, stamps added by the user from a stamp store, etc. For example, the candidate image list may include at least one stamp favorited by the user.

[0041] At this time, if a user's image selection operation for a candidate image in the candidate image list is detected, the candidate image selected by the user is determined as an input image, and a search result list is displayed.

[0042] For example, as shown in FIG. 1c, after a user triggers the display of a stamp selection panel at the bottom of an image selection page 1c2 by operating an image selection control, i.e., a "+" plus sign button, in an image search page 1c1, the user may select "Stamp 1" from the stamp selection panel as an input stamp by a click operation. When the user clicks to select "Stamp 1", a search result list is displayed, which includes stamps similar to "Stamp 1", such as Stamp 11, Stamp 12, and so on.

[0043] (2) The user acquires an input image by taking a photograph.

[0044] Among them, the image selection page may include an image shooting control, and in this case, when a user's trigger operation on the image shooting control is detected, an image shooting page is displayed, and based on the user's shooting operation on the image shooting page, a shooting result page is displayed, and the shooting result page includes a shooting result image and an image confirmation control, and when a user's confirmation operation on the image confirmation control is detected, the shooting result image is determined as the input image.

[0045] The image capture may be a photograph or a video.

[0046] For example, as shown in FIG. 2a, taking an image as a stamp, the stamp selection panel at the bottom of the stamp selection page 2a1 may further include an image capture control, such as a "take" button. If the user wants to select an input stamp by taking a picture (i.e., the taken image is used as the input stamp), the user may perform an operation, such as a click, slide, etc., on the "take" button. At this time, the terminal calls the camera and displays the image capture page 2a2, and the user may then operate on the image capture page 2a2 to take a required image. For example, the user may click the capture button on the image capture page 2a2, and the terminal displays the capture result page 2a3, which includes an image obtained by taking a picture, such as image A, and an image confirmation control "add" button. If the user wants to use the image A obtained by taking a picture as the input stamp, the user may perform an operation on the "add" button. At this time, the terminal uses the image A obtained by taking a picture as the input stamp, and displays a search result list, which includes similar stamps similar to the image A, such as similar stamp A1, similar stamp A2, etc. In one embodiment, when the user operates on the "Add" button, the terminal may further add image A to the user's stamp library, for example, marking image A as a favorite.

[0047] (3) The user selects a photo from the local album as the input image.

[0048] Wherein, the image selection page may include a photo selection control, in which, when a user's trigger operation on the photo selection control is detected, a photo selection page is displayed, the photo selection page includes at least one photo in a local album, and according to the user's photo selection operation on the photo selection page, the photo selection page is marked with a photo selected by the user, and a photo confirmation page is displayed, the photo confirmation page includes a photo confirmation control, and when a user's confirmation operation on the photo confirmation control is detected, the photo selected by the user is determined as an input image.

[0049] For example, as shown in FIG. 2b, the stamp selection surface at the bottom of the stamp selection page 2b1 may further include a photo selection control, such as a "Photo" button. If the user wants to select a photo from a local album as an input stamp, the user may perform an operation on the "Photo" button, such as a click, slide, etc. At this time, the terminal displays a photo selection page 2b2, which includes at least one photo in the local album. The user may select a photo on the photo selection page 2b2 by an operation, and the terminal may mark the photo selected by the user. For example, if the user selects photo B1 by a click operation, the user may check the photo B1, and if the user wants to use the selected photo B1 as an input stamp, the user may click a photo confirmation control, such as an "OK" button, in the photo confirmation page 2b3. At this time, the terminal uses the photo selected by the user as the input stamp, and displays a search result list, which includes similar stamps similar to the photo B1, such as similar stamp B11 and similar stamp B12. In one embodiment, if the user operates on the "OK" button, the terminal may further add the photo B1 to the user's stamp library, for example, favorite the image B1.

[0050] (4) The user determines the input image by voice.

[0051] Wherein, the image selection page may include a voice selection control, in which, when a trigger operation of the user on the voice selection control is detected, a voice input page is displayed, the voice input page includes a voice input control, when a voice input operation of the user on the voice input control is detected, voice information input by the user is collected, when a voice input end operation of the user on the voice input control is detected, a candidate image matching the voice information is displayed, and when a selection operation of the user on the candidate image is detected, the candidate image selected by the user is determined as an input image.

[0052] Wherein, the candidate images may be candidate images that match the voice information in the local database of the terminal, for example, the candidate images that match the voice information in the local stamp library. Wherein, matching of an image with voice information may include matching of the text content in the image with the voice content, matching of the meaning or attribute content of the image with the voice content, etc.

[0053] For example, as shown in FIG. 2c, the stamp selection panel at the bottom of the image selection page 2c1 may further include a voice selection control, such as a “voice input” button. When the user wants to select an image as an input stamp by voice, the user may perform an operation, such as a click, slide, etc., on the “voice input” button, and at this time, the terminal displays a voice input page 2c2, and the voice input page 2c2 includes a voice input control, such as a microphone icon. The user may perform an operation on the microphone icon in the voice input page 2c2, for example, the user may long press the microphone icon to trigger the collection of the voice information input by the user, and when the user ends the long press operation, the terminal displays one stamp selection page 2c3, and the stamp selection page 2c3 includes candidate stamps matching the voice information in the local stamp library, such as stamp C1, stamp C2, stamp C3, etc., and the user may select one candidate stamp as an input stamp on the stamp selection page 2c3, for example, the user may select the candidate stamp C1 by clicking. At this time, the terminal displays a search result list, which includes similar stamps similar to the stamp C1, such as the similar stamp C11 and the similar stamp C12.

[0054] (5) The user determines the input image by image rendering.

[0055] Wherein, the image selection page may include an image rendering control, in which, when a user's trigger operation on the image rendering control is detected, an image rendering page is displayed, the image rendering page includes an image rendering area and a rendering image confirmation control, and according to the user's rendering operation in the image rendering area, the image rendering page displays the user's rendered image, and when a confirmation operation on the rendering image confirmation control is detected, the image rendering page is determined as the input image.

[0056] For example, as shown in FIG. 2d, the stamp selection panel at the bottom of the image selection page 2d1 may further include a rendering control, such as a “paintbrush”-shaped icon. When the user wants to determine an input stamp by image rendering, the user may perform an operation on the “rendering” icon, such as clicking, sliding, etc. At this time, the terminal displays a stamp rendering page 2d2, which includes a stamp rendering area and a rendering image confirmation control, such as an “add” button. The user may perform a rendering operation in the stamp rendering area, such as the user may render a required stamp by adopting corresponding colors, styles, and lines in the stamp rendering area. After the user finishes rendering the stamp G, the user may perform an operation on the rendering image confirmation control, such as an “add” button. At this time, the terminal takes the stamp rendered by the user as the input stamp, and displays a search result list, which includes similar stamps similar to the stamp G, such as similar stamp G1 and similar stamp G2. In one embodiment, when the user operates on the "Add" button, the terminal may further add the rendered image G to the user's stamp library, for example, favorite the image G.

[0057] The above input image selection methods can be arbitrarily combined, so that the user can adopt multiple methods to select input images for searching. For example, the stamp selection panel may include at least one of candidate stamps, an image capture control, a photo selection control, an audio selection control, and an image rendering control.

[0058] In one embodiment, considering that image search requires a user to wait for a certain period of time, in order to avoid the waiting causing the user impatience and degrading the user experience, a search waiting page is further displayed. Therefore, the method further includes a step of displaying an input image and a search result loading icon on a search waiting page of the instant messaging client, and step 105 of "generating and displaying a search result list according to the at least one output image" specifically includes a step of displaying the search result list on a search result page if the search is successful.

[0059] For example, as shown in FIG. 1c, after a user selects an input stamp, the terminal displays a stamp search waiting page 1c3, which includes the input stamp selected or determined by the user and a search result loading icon, which can be set according to actual needs, such as a circular loading icon, a long loading icon (e.g., a progress bar), etc.

[0060] In particular, in embodiments of the present application, the search result list may include output images that are similar to the input image along at least one (e.g., one or more) image similarity dimensions. For example, the search result list may include output stamps that are similar to the input stamp along at least one stamp similarity dimension, etc.

[0061] In one embodiment of the present application, in step 103, "pre-setting a plurality of image similarity dimensions according to the level of similarity between the two chat dialogue images" may specifically include a step of pre-obtaining a plurality of chat dialogue images to be used in the instant messaging client, and a step of hierarchizing information contained in the plurality of chat dialogue images, and setting the plurality of image similarity dimensions to any one of a character content dimension, a semantic dimension, a role dimension, a movement dimension, and a conversation relationship dimension.

[0062] Among them, the image similarity dimension is the similarity type, similarity manner, or similarity level between two chat dialogue images, etc. In the embodiments of the present application, the similarity between images includes that the images are the same and / or similar, and specifically can be selected according to actual needs.

[0063] Among them, the image similarity dimension can be divided into multiple types according to actual needs, that is, the similarity level between two chat dialogue images may be multiple types, such as image similarity, semantic similarity, conversation relationship similarity, etc. For example, the similarity between two chat dialogue images may be image similarity, semantic similarity, or conversation relationship similarity, etc.

[0064] For example, the image similarity dimensions may include, but are not limited to, the following dimensions:

[0065] Textual content dimension of images: For example, the text in two chat dialogue images is the same or similar. Semantic dimension of images: For example, the meaning expressed by two chat dialogue images is the same or similar. Role dimension of images: For example, the roles in two chat interaction images are the same or similar. Movement dimension of roles in images: For example, the movements performed by roles in two chat interaction images are the same or similar. Conversational relationship dimension in images: This includes the fact that the conversational relationship of the text content is similar, or the meaning expressed by the image exhibits a similar conversational relationship, for example, the text content in two chats forms a conversational relationship.

[0066] For example, taking a stamp as an example, the stamp similarity dimensions may include, but are not limited to, the following dimensions:

[0067] The textual content dimension of the stamps: for example, the text on two stamps is the same or similar. Semantic dimension of stamps: For example, the meanings expressed by two stamps are the same or similar, e.g., both stamps are stamps expressing "hello", or one expresses "hello" and the other expresses "hello". Stamp role dimension: For example, the roles in two stamps are the same or similar, for example, the virtual characters in the two stamps are both "XX Baby" cartoon characters. Movement dimension of the role of the stamp: For example, the movements made by the roles in two stamps are the same or similar, for example, the hands of the roles in the two stamps are both making a gesture of "victory", or the role in one stamp is making the movement of "covering the face with hands" and the other stamp is making the movement of "covering the mouth". Stamp conversational relationship dimension: This includes the similarity of the conversational relationship of the text content. For example, the text content in two stamps forms a conversational relationship. For example, the text content of one stamp is "Are you sick?" and the text content of the other stamp is "Do you have any medicine?", and these two stamps have a similar conversational relationship.

[0068] For example, a search for output images similar to the input image across multiple image similarity dimensions may be triggered to obtain a set of output images, where the set of output images may include output images similar to the input image across each or various image similarity dimensions. In one embodiment, there may be multiple triggers for searching output images, for example, a search for output images similar to the input image may be triggered after an input image is acquired, or a search for output images similar to the input image may be triggered simultaneously with an additional input image being acquired.

[0069] For example, in one embodiment, a search image may be triggered when a user performs a selection operation on a candidate image list, and specifically, may include triggering a search for output images similar to an input image across multiple image similarity dimensions when a user selection operation on a candidate image in the candidate image list is detected.

[0070] Also, for example, when a user's confirmation operation on an image confirmation control is detected, it triggers a search for output images similar to the input image in multiple image similarity dimensions.

[0071] Also, for example, when a user's confirmation action on a photo confirmation control is detected, it triggers a search for output images similar to the input image across multiple image similarity dimensions.

[0072] Also, for example, when a user's selection operation on a candidate image is detected, a search for output images similar to the input image in multiple image similarity dimensions is triggered.

[0073] Also, for example, when a confirmation operation on a rendered image confirmation control is detected, it triggers a search for output images similar to the input image in multiple image similarity dimensions.

[0074] For example, as shown in FIG. 1c, after displaying the stamp selection panel, the user may select “Stamp 1” from the stamp selection panel as an input stamp by clicking. When the user clicks to select “Stamp 1”, it triggers the terminal to search for output stamps similar to “Stamp 1” in multiple stamp similarity dimensions.

[0075] In the embodiments of the present application, there are multiple ways to trigger the search for output images. For example, in one embodiment, the terminal may itself trigger the search for output images similar to the input image in multiple image similarity dimensions.

[0076] Specifically, step 104 "searching for at least one output image similar to said input image in said plurality of image similarity dimensions" may include performing the following process for each image similarity dimension.

[0077] First feature information in the image similarity dimension of the input image is extracted, second feature information in the image similarity dimension of each candidate image in an image database is obtained, a similarity between the first feature information and each second feature information is calculated, and a candidate image in the image database that is similar to the input image in the image similarity dimension is determined as the output image according to each calculated similarity.

[0078] In this manner, the terminal is used to obtain output images from the server and generate a search result list.

[0079] Among them, the feature information in each or various image similarity dimensions represents information such as attributes in the image similarity dimension, and is used to calculate similarity information between two images in the image similarity dimension.

[0080] For example, the feature information in the image similarity dimension may include feature information such as text-like feature information and vector-like feature information corresponding to the content or meaning of the image. Among them, the text-like feature information may include character feature information and image type information (e.g., classification tag information), etc. Among them, the character feature information may be obtained by adopting character recognition, and the image type information may be obtained by classifying the image. Among them, when the vector-like feature information is extracted by adopting a feature extraction model, it may be an eigenvector (e.g., a multidimensional eigenvector) output by the model. The feature extraction model may be a deep learning model, such as a residual network model (ResNet), a face recognition network model (FaceNet), etc.

[0081] In one embodiment, the step 105 of “generating and displaying a search result list according to the at least one output image” may specifically include extracting third feature information of each output image, and sorting each output image according to the third feature information to obtain the search result list.

[0082] Wherein, when the output image is determined from the candidate images in the image database, the third feature information is the same as the second feature information.

[0083] In an embodiment of the present application, searching for output images in at least one image similarity dimension may include searching for output images similar to the input image in multiple image similarity dimensions simultaneously, e.g., searching for output images similar to the input image in image similarity dimension 1, image similarity dimension 2, and image similarity dimension 3 simultaneously, and may further include searching for output images similar to the input image in each image similarity dimension respectively, e.g., searching for output images similar to the input image in image similarity dimension 1, searching for output images similar to the input image in image similarity dimension 2, ... searching for output images similar to the input image in image similarity dimension n, where n is a positive integer greater than 2.

[0084] Taking stamps as an example, searching for output stamps similar to an input stamp in at least one stamp similarity dimension may include simultaneously searching for output stamps similar to the input stamp in multiple stamp similarity dimensions, for example, stamp 11 is similar to stamp 1 in image, meaning, and conversational relationship, in which case stamp 11 is similar to input stamp 1 in three dimensions at the same time. It may also include searching for output stamps similar to the input stamp in each stamp similarity dimension, for example, stamp 11 is similar to stamp 1 in meaning, stamp 12 is similar to stamp 1 in role, stamp 13 is similar to stamp 1 in conversational relationship, etc.

[0085] In one embodiment, the search result list may include output images that are similar to the input image along each or various image similarity dimensions, e.g., output stamps that are similar to the input stamp along each dimension of image similarity, semantic similarity, conversational relationship similarity, etc.

[0086] In practical applications, the search results may be displayed by pages, specifically, if the search results are successfully obtained, a search result page is displayed, which includes the input image and a search result list. For example, as shown in FIG. 1c, if the terminal successfully obtains the search results, a stamp search result page 1c4 is displayed, which includes the input stamp 1 and output stamps similar to the input stamp 1 in various stamp similarity dimensions, such as stamp 11, stamp 12, etc.

[0087] In one embodiment, the user may be allowed to favorite the selected output image by operating the search result list so that the user can easily use the searched output image. Specifically, the image search method further includes the steps of: displaying an image operation page of the target output image when a user's confirm operation on the target output image in the search result list is detected, the image operation page including an image add control; and adding the target output image to a chat image library of the instant communication client when a user's add operation on the image add control is detected.

[0088] In one embodiment, in order to allow the user to easily use the output image searched for by the user, the user may be allowed to directly use the selected output image through operation in the search result list. Specifically, the image search method further includes the steps of: when a user's confirm operation on a target output image in the search result list is detected, displaying an image operation page of the target output image, the image operation page including an image transmission control; displaying a candidate object list when a user's image sending operation on the image sending control is detected, the candidate object list including at least one candidate chat partner; obtaining a target chat partner selected by the user from the candidate object list; and Sending the target output image to the target chat partner is included.

[0089] Among them, the potential chat partners may include single chat partners, e.g., a user, group chat partners, e.g., a group, and so on.

[0090] For example, as shown in FIG. 1f, after displaying the stamp search result list, the user may select a target output stamp by operating the output stamps in the list, and when the user performs an operation, such as clicking, sliding, etc., on an output stamp in the list, for example, output stamp 12, the terminal may display a stamp details page of the output stamp, for example, the stamp details page of stamp 12, on the image operation page 1f1, which includes a stamp sending control, for example, a “Send to a Friend” button, and a stamp adding control, for example, an “Add to Stamp” button.

[0091] If the user wishes to add the output stamp to their favorites, they may click the "Add to Stamp" button, at which point the terminal will add the stamp 12 to the client's user stamp library for future use by the user.

[0092] If the user wants to directly send the output stamp for use, he / she may click the "Send to Friends" button, at which time the terminal may display a candidate chat partner list page 1f2 for "select one chat", the list may include single chat partners and group chat partners, etc., and the user may operate on the object list page 1f2 to select one target chat partner. For example, if the chat partner is a single chat partner, the stamp 12 may be sent to the chat conversation of the single chat partner, as shown on page 1f3, and if the chat partner is a group chat partner, the stamp 12 may be sent to the group chat conversation corresponding to the group chat partner.

[0093] As is clear from the above, the embodiment of the present application searches for the output image based on the image, that is, searches for the output image by the image (for example, searches for the stamp by the stamp), and realizes the search for the output image. The user does not need to input text information to search for the image, which simplifies the image search process and improves the efficiency of the image search. In addition, the information contained in the image is richer than the method of searching for the output image by the image and searching for the image by the text, and the search needs of the user, such as the content, format, etc. of the image, can be better expressed, and the output image desired by the user can be more accurately searched.

[0094] In addition, the scheme can further search output images in multiple image similarity dimensions, making the searched images more comprehensive and improving the probability of matching the user's needs, thus improving the image search scope and accuracy.

[0095] In an embodiment, the method may further trigger another device, for example, a server, to search for output images similar to the input image in multiple image similarity dimensions, and then obtain the searched output images from another device, for example, a server. Figure 2f is a flowchart of an image search method provided in another embodiment of the present application, which is executed by a server, for example, the server 120 in Figure 1a. Specifically, the image search method includes the following steps:

[0096] 401: Based on an image search request sent from a terminal, an input image for search is determined.

[0097] 402: Pre-set a number of image similarity dimensions according to the similarity level between two chat dialogue images.

[0098] 403: Search and obtain at least one output image similar to the input image in a plurality of image similarity dimensions, where each output image is similar to the input image in at least one image similarity dimension.

[0099] 404: Generate a search result list according to the at least one output image.

[0100] 405: The search result list is returned to the terminal, which displays the list on the interface of the instant messaging client.

[0101] Among them, the specific implementation method of steps 402 to 404 is the same as that of steps 103 to 105 above, and thus will not be described in detail here.

[0102] The methods described in the above embodiments will now be described in more detail with reference to the following examples.

[0103] In this embodiment, the first image search device is specifically integrated in a terminal, and the second image search device is specifically integrated in a server. As shown in Fig. 3a, the chat dialogue image method can be applied to the terminal 110 and the server 120 shown in Fig. 1a, and the specific process is as follows:

[0104] 301: The terminal displays a content search page of an instant messaging client, where the content search page includes an image search control.

[0105] 302: When a user's trigger operation on the image search control is detected, the terminal displays an image search page of the instant messaging client, where the image search page includes an image selection control.

[0106] 303: If a user trigger operation on the image selection control is detected, the terminal displays an image selection page.

[0107] 304: The terminal determines an input image based on the user's input image determination operation on the image selection page.

[0108] For the specific implementation of each of the above steps, please refer to the description of the above embodiments.

[0109] Also, for example, as shown in FIG. 3b, an architecture of a stamp search system provided by an embodiment of the present application, when a user selects an input stamp, the instant messaging client of the terminal sends an image search request to an access layer, for example, an access layer server, and the access layer may forward the image search request to a similar stamp search service, for example, to a similar stamp search server. The image search request may carry an input image, for example, a stamp, and in one embodiment, an image identifier of the image may be used to represent the image to perform image search, in order to improve the efficiency of image search. For example, the instant messaging client may send an image search request carrying a stamp identifier (id) to the access layer.

[0110] 305: The server determines input images for the search based on the image search request sent from the terminal.

[0111] For example, the server may parse the image search request and obtain the input image or an image identifier for the input image, such as the stamp id, to be searched.

[0112] 306: The server extracts first feature information in multiple image similarity dimensions of the input image.

[0113] For example, as shown in Fig. 3b, a feature service module may be adopted to extract feature information of an input stamp in multiple stamp similarity dimensions, where the feature service module may be realized by another server as the function of the feature server module, or the feature service module is a local service module of the server. Specifically, the description of the image similarity dimension may refer to the description of the above embodiment.

[0114] For example, stamp similarity dimensions or scenes have multiple types, such as image similarity, semantic similarity, and conversational relationship. In order to meet the needs of multiple types of scenes, it is necessary to extract features of multiple dimensions, including text-like features obtained by character recognition, text-like features from stamp tag recognition, and vector-like features (e.g., multi-dimensional eigenvectors output by models such as ResNet and FaceNet), etc.

[0115] In one embodiment, in order to improve the efficiency of feature extraction and image search, some image feature information (which may be called offline feature information) may be extracted in advance and stored in a database (which may be called an offline feature library). When searching for an image, if the input image exists in the database, it may be directly extracted from the database.

[0116] Specifically, when an input image exists in a specified feature set, feature information in multiple image similarity dimensions corresponding to the input image is extracted from the specified feature set, in which the specified feature set includes a specified image and feature information in multiple image similarity dimensions corresponding to the specified image.

[0117] If the input image does not exist in the predetermined feature set, multi-dimensional feature extraction is performed on the input image to obtain feature information of the input image in multiple image similarity dimensions, where the representation format of the predetermined feature set can be a database, for example, an offline feature library.

[0118] In one embodiment, the feature information in multiple image similarity dimensions may include text-like feature information and vector-like feature information. If the input image does not exist in a predetermined feature set, the following method may be adopted to extract features.

[0119] Perform character recognition on the input image to obtain a character recognition result, and construct a first text-like feature information of the input image according to the character recognition result; Perform image classification on the input image to obtain an image classification result, and construct second text-like feature information of the input image according to the image classification result; Based on the feature extraction model, the eigenvectors of the input image are extracted, and the vector-class features of the input image are obtained.

[0120] For example, as shown in Fig. 3b, the data in the offline feature library is periodically updated to the data service module for online use, and the feature service module extracts features according to the feature extraction method described above. For example, as shown in Fig. 3c, first, detect whether the input stamp hits the offline feature library, that is, whether the feature information of the input stamp exists in the offline feature library. If YES, execute the offline feature extraction of the input stamp in multiple stamp similar dimensions from the offline feature library; if not, obtain the input stamp image, and respectively perform multi-dimensional feature extraction such as text-like feature extraction and vector-like feature extraction on the input stamp image to obtain the multi-dimensional online feature of the input stamp.

[0121] 307: The server searches for output images similar to the input image based on the first feature information in multiple image similarity dimensions.

[0122] The retrieved output images may be combined to form an output image set, the output image set including output images that are similar to the input image along at least one image similarity dimension.

[0123] For example, the server may search an image database for images similar to the input image based on the feature information in each image similarity dimension, where the image database, e.g., a stamp library, may include a number of images, e.g., stamps, that can be provided to a user.

[0124] In one embodiment, in order to improve the accuracy and richness of the output image search, further online output images may be supplemented, for example, output images of some images, for example, an output image list, may be pre-set and stored in an output image database, and when searching for an output image, the output image may be searched for in the output image database to supplement the search results.

[0125] Specifically, based on the first feature information in each image similarity dimension, search for output images similar to the input image to obtain a first output image subset, where the first output image subset includes output images similar to the input image in at least one image similarity dimension; obtain a predetermined output image corresponding to the input image based on the output image mapping relationship set, and obtain a second output image subset, where the output image mapping relationship set includes a mapping relationship between the input image and the predetermined output image; and combine the first output image subset and the second output image subset to obtain an output image set.

[0126] Among them, the set of output image mapping relationships may be stored in a database, which may be called an output image library, for example, an output stamp library.

[0127] For example, as shown in Fig. 3b, after the feature service module extracts multiple dimension feature information of the stamp, the recall service module may recall the output stamp when searching according to the extracted multiple dimension feature information. Specifically, as shown in Fig. 3d, the output stamp may be searched in a similar stamp library, where the similar stamp library is a similar stamp list calculated offline, which mines multiple dimensions of similar information and supplements the online recall result, and the output stamp may be searched in the stamp library according to text-like features (i.e., text recall), for example, by a text search engine to realize the text recall, and the output stamp may be searched in the stamp library according to vector-like features (i.e., vector recall), for example, by a vector search engine to realize the vector recall, and then the multiple recalled output stamps are combined to obtain a searched output stamp set.

[0128] In one embodiment, there are multiple ways to search for an output image in an image database based on feature information in an image similarity dimension, which may include, for example, extracting, for each image similarity dimension, first feature information of the input image in the image similarity dimension, obtaining second feature information of each candidate image in the image database in the image similarity dimension, calculating similarities between the first feature information and each second feature information, and determining, as the output image, a candidate image in the image database that is similar to the input image in the image similarity dimension according to each calculated similarity. For example, an image in the image database whose similarity is greater than a predetermined similarity threshold is selected as the output image of the input image.

[0129] 308: The server extracts third feature information of each output image.

[0130] Wherein, the third feature information of the output image may include feature information of the output image in one or more image similarity dimensions.

[0131] For example, the server may extract feature information of the output image from a predefined feature set, e.g., extract the output image in a similar dimension from an offline feature library.

[0132] 309: The server sorts each output image according to the third feature information, and obtains a search result list according to the sorting result.

[0133] For example, as shown in Fig. 3b and Fig. 3e, after the recall service module obtains multiple recalled output stamps, the sort service module may sort the multiple recalled output stamps, for example, by extracting offline feature information (offline feature information in multiple image similarity dimensions) of the output stamps from an offline feature library, and then performing duplicate deduplicating of the stamps and sorting the recalled output stamps based on the offline feature information in multiple image similarity dimensions of the output stamps, for example, by integrating various stamps for scoring and sorting.

[0134] In the embodiment of the present application, the searched output images can be sorted based on the multidimensional feature information of the searched output images, and by sorting the multidimensional feature information of the output images, the user's needs for image similarity dimensions can be expressed, i.e., the user is looking for an output image that is dimensionally similar to the input image, and thus the output images that meet the user's needs are sorted to the top, thereby improving the search accuracy of the output images.

[0135] For example, in one use scenario, when everyone in a group chat expresses congratulations, gratitude, or agreement on something, it avoids the embarrassment of sending the same stamp as everyone else, and makes others feel that they are serious about this matter and not inappropriate. For example, when someone in a group celebrates a birthday, everyone sends a happy birthday stamp, but the happy birthday stamp is not commonly used, and there may be relatively few stamps that the user has favorited, so sending the stamp sent in the previous chat will feel very inappropriate, so the user needs to find a different stamp to send to the other person. See FIG. 4a, which shows that the method provided by the embodiment of the present application can obtain a happy birthday stamp search result. In FIG. 4a, in this use scenario, the user attaches importance to the text content of the stamp and wants to search for the output stamp of "happy birthday", so after employing the method of the embodiment of the present application to search the output stamp list, the multi-dimensional features of the stamp, such as text features, may be output, and the output stamps in the list may be sorted, so that the stamp containing the text "happy birthday" is sorted higher in the list than other similar stamps.

[0136] In another use scenario of output stamps, when a user sees a stamp, he / she likes the stamp itself very much and wants to find more similar stamps, the output stamp can help the user quickly find similar stamps. For example, when a user receives a stamp of "XX Baby" and wants to get more stamps related to "XX Baby", multiple similar output stamps can be quickly obtained by searching, and the user does not need to input characters to search.

[0137] For example, in Fig. 4b, in this usage scenario, the user values ​​the character role of the stamp and wants to search for an output stamp whose character role is "XX Baby". Therefore, after searching the output stamp list using the method of the embodiment of the present application, the multi-dimensional features of the stamp, such as character features (stamp tag features), may be output, and the output stamps in the list may be sorted so that the stamp whose character role is "XX Baby" is sorted higher in the list than other output stamps.

[0138] In another use scenario of output stamps, when a user receives a stamp, the user may wish to provide an interesting stamp feedback result as the output stamp to increase the fun of sending stamps during chat. For example, when the user receives a stamp "Are you sick?", the user may provide a stamp with a contentious meaning such as "Are you sick?" or "Do you have medicine?" as the output stamp to send to the other person, thereby increasing the fun of chatting.

[0139] For example, in Fig. 4c, in this usage scenario, the user attaches importance to the conversational relationship of stamps and wants to search for output stamps that have a conversational relationship with the input stamp. Therefore, after searching the output stamp list using the method of the embodiment of the present application, multi-dimensional features of stamps, such as character-like features and vector-like features, are output, and the output stamps in the list are sorted, and the output stamps that have a conversational relationship with the input stamp are sorted to the top of the list, relative to other output stamps.

[0140] 310: The server transmits the sorted search result list to the terminal.

[0141] In one embodiment, depending on the number of images that need to be displayed at the terminal, the server may select a corresponding number of images from the sorted output image set and send them to the terminal.

[0142] 311: The terminal displays the search result list.

[0143] As is clear from the above, the embodiment of the present application can search for output images based on images. That is, output images are searched by images (for example, stamps are searched by stamps) to realize search for similar chat images. In addition, the user does not need to input text information to search for images, which simplifies the image search process and improves the efficiency of image search. In addition, compared with the method of searching for output images by images and searching for images by text, the information contained in the image is richer, and the user's search needs, such as image content, format, etc., can be better expressed, and the user's desired output image can be more accurately searched for.

[0144] In addition, the scheme can further search output images in multiple image similarity dimensions, making the searched images more comprehensive and improving the probability of matching the user's needs, thus improving the image search scope and accuracy.

[0145] To better implement the above method, the embodiment of the present application also provides an image search device (ie, a first image search device), where the first image search device may be integrated into a terminal. For example, as shown in FIG. 5a, the image search device 500 can be applied to the terminal 110 shown in FIG. 1a, and includes a first display unit 501, a determination unit 502, a setting unit 503, a search unit 504, a generation unit 505, and a second display unit 506, wherein the first display unit 501 is used for displaying an image selection page of a client, the determination unit 502 is used for determining an input image for search based on an operation performed by a user on the image selection page, the setting unit 503 is used for pre-setting a plurality of image similarity dimensions according to a similarity level between two chat dialogue images, the search unit 504 is used for searching and obtaining at least one output image similar to the input image in the plurality of image similarity dimensions, wherein each output image is similar to the input image in at least one image similarity dimension, the generation unit 505 is used for generating a search result list according to the at least one output image, and the second display unit 506 is used for displaying the search result list.

[0146] In one embodiment, the setting unit 503 is used to pre-acquire a plurality of chat dialogue images used by the instant messaging client, perform hierarchical classification on the information contained in the plurality of chat dialogue images, and set the plurality of image similarity dimensions to any one of a character content dimension, a meaning dimension, a role dimension, a movement dimension, and a conversation relationship dimension.

[0147] In one embodiment, the first display unit 501 is further used for displaying the input image and a search result loading icon on a search waiting page of the instant messaging client, and the second display unit 506 is used for displaying the search result list on a search result page of the instant messaging client if the search is successful.

[0148] In one embodiment, the first display unit 501 further displays a chat conversation list page of the instant messaging client, the chat conversation list page including a content search control, and is used to display the content search page when a user triggers an operation of the content search control.

[0149] In one embodiment, the first display unit 501 further displays a function page of the instant messaging client, the function page including a search function control, and is used to display the content search page when a user trigger operation on the search function control is detected.

[0150] In one embodiment, the search unit 504 is used to perform the following process for each image similarity dimension: extract first feature information of the input image in the image similarity dimension, obtain second feature information of each candidate image in the image database in the image similarity dimension, respectively calculate similarities between the first feature information and each second feature information, and determine a candidate image in the image database that is similar to the input image in the image similarity dimension as the output image according to each calculated similarity.

[0151] In one embodiment, the feature information includes text-like feature information and / or vector-like feature information.

[0152] In one embodiment, the generating unit 505 is used to extract third feature information of each output image, sort each output image according to the third feature information, and obtain the search result list according to the sorting result.

[0153] In one embodiment, the second display unit 506 further displays an image operation page of the target output image when a user's confirm operation on the target output image in the search result list is detected, the image operation page including an image add control, which is used to add the target output image to a chat image library of the instant messaging client when a user's add operation on the image add control is detected.

[0154] In the embodiment shown in FIG. 5b, the image retrieval device 500 further includes a third display unit 507, a fourth display unit 508, a user interface unit 509, and a sending unit 510, wherein the third display unit 507 is used for displaying an image operation page of the target output image when a user's confirmation operation on a target output image in the search result list is detected, the image operation page including an image sending control, the fourth display unit 508 is used for displaying a candidate object list when a user's image sending operation on the image sending control is detected, the candidate object list including at least one candidate chat partner, the user interface unit 509 is used for obtaining a target chat partner selected by a user from the candidate object list, and the sending unit 510 is used for sending the target output image to the target chat partner.

[0155] To better implement the above method, the embodiment of the present application also provides an image search device (ie, a second image search device), where the second image search device may be integrated into a server. For example, as shown in FIG. 6a, the image search apparatus 600 can be applied to the server 120 in FIG. 1a, and includes a determining unit 601, a setting unit 602, a searching unit 603, a generating unit 604, and a sending unit 605, wherein the determining unit 601 is used for determining an input image according to an image search request sent from a terminal, the setting unit 602 is used for pre-setting a plurality of image similarity dimensions according to the similarity level between two chat dialogue images, the searching unit 603 is used for searching and obtaining at least one output image similar to the input image in the plurality of image similarity dimensions, wherein each output image is similar to the input image in at least one image similarity dimension, the generating unit 604 is used for generating a search result list according to the at least one output image, and the sending unit 605 is used for returning the search result list to the terminal for displaying on the interface of an instant messaging client on the terminal.

[0156] In one embodiment, the generation unit 604 includes an extraction subunit 6041 and a sorting subunit 6042, where the extraction subunit 6041 is used for extracting third feature information of each output image, and the sorting subunit 6042 is used for sorting each output image according to the third feature information, and obtaining the search result list according to the sorting result.

[0157] In addition, the embodiment of the present application further provides a computer device, which may be a terminal or a server. As shown in FIG. 7, it shows a structural schematic diagram of a computer device according to the embodiment of the present application, and specifically, the computer device may include components such as a processor 701 of one or more processing cores, a memory 702 of one or more computer-readable storage media, a power supply 703, and an input unit 704. As can be understood by those skilled in the art, the structure of the computer device shown in FIG. 7 does not limit the computer device, and the computer device may include more or less components than those shown, or may combine some components or have different component arrangements. Among them, the processor 701 is the control center of the computer device, which uses various interfaces and wirings to connect each part of the entire computer device, calculates or executes software programs and / or modules stored in the memory 702, and calls up data stored in the memory 702 to perform various functions and data processing of the computer device, thereby monitoring the entire computer device. Optionally, the processor 701 may include one or more processing cores, and preferably, the processor 701 may integrate an application processor and a modulation / demodulation processor, in which the application processor mainly processes the operating system, user interface, application programs, etc., and the modulation / demodulation processor mainly processes wireless communication. It can be understood that the modulation / demodulation processor does not have to be integrated into the processor 701.

[0158] The memory 702 may be used to store software programs and modules, and the processor 701 operates the software programs and modules stored in the memory 702 to execute various functional applications and data processing. The memory 702 may mainly include a program storage area and a data storage area, in which the program storage area may store an operating system, an application program required for at least one function (e.g., a voice playback function, an image playback function, etc.), and the data storage area may store data generated according to the use of the computer device, etc. The memory 702 may also include a high-speed random access memory, and may further include a non-volatile memory, such as at least one magnetic disk storage device, a flash device, or other volatile solid-state storage device. Similarly, the memory 702 may further include a memory controller, thereby providing the processor 701 access to the memory 702.

[0159] The computer device further includes a power supply 703 for powering each component, and preferably, the power supply 703 may be logically connected to the processor 701 through a power management system, thereby realizing functions such as charging, discharging management, and power management through the power management system. The power supply 703 may further include any unit such as one or more DC or AC power sources, a recharging system, a power failure detection circuit, a power converter or inverter, and a power status indicator.

[0160] The computing device may further include an input unit 704, which may be used to receive input numeric or textual information and generate keyboard, mouse, joystick, optical or trackball signal inputs associated with user settings and function control.

[0161] Although not shown, the computer device may further include a display unit, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 701 in the computer device loads executable files corresponding to the processes of one or more application programs into the memory 702 according to the following instructions, and executes the application programs stored in the memory 702 by the processor 701, thereby achieving various functions.

[0162] For the specific implementation of each of the above operations, please refer to the above embodiments, and detailed description will not be given here.

[0163] As can be understood by those skilled in the art, all or some of the steps of the various methods in the above embodiments may be completed by instructions or may be completed by controlling related hardware by instructions, and the instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0164] To this end, embodiments of the present application further provide a storage medium, on which a plurality of instructions are stored, the instructions being loadable by a processor to thereby perform steps of any of the image retrieval methods provided by embodiments of the present application.

[0165] The storage medium may include a read only memory (ROM), a random access memory bank (RAM), a magnetic disk, an optical disk, and so on.

[0166] The instructions stored in the storage medium can execute any steps of the image search method provided by the embodiments of the present application, and thus can realize the beneficial effects realized by any image search method provided by the embodiments of the present application, and the details can be referred to the above embodiments, so that a detailed description is not provided here.

[0167] The image search method, device, computer device and storage medium provided by the embodiments of the present application have been described in detail above, and the principles and embodiments of the present application have been described by applying specific examples. However, the description of the above embodiments is merely intended to help understand the method and core idea of ​​the present application. In addition, those skilled in the art may make modifications to specific embodiments and application scope based on the idea of ​​the present application. As such, it should be understood that the contents of this specification do not limit the present application. [Explanation of symbols]

[0168] 100 Image Search System 110 Terminal 111 Image Search Device 120 Servers 500 Image search device 501 1st display unit 502 Decision Unit 503 Setting Unit 504 Search Unit 505 Generator Unit 506 Second display unit 507 Third display unit 508 4th display unit 509 User Interface Unit 510 Transmitter Unit 600 Image Search Device 601 Decision Unit 602 Setting Unit 603 Search Unit 604 Generator Unit 605 Transmitter Unit 701 Processor 702 Memory 703 Power supply 704 Input Unit

Claims

1. An image search method, executed by a terminal, comprising: displaying chat dialogue images received in the chat conversation by the instant messaging client; determining the chat dialogue image as an input image for a search; presetting a number of image similarity dimensions according to a similarity level between two chat dialogue images; searching for at least one output image similar to the input image in the plurality of image similarity dimensions, each output image being similar to the input image in at least one image similarity dimension; and generating and displaying a search result list in response to the at least one output image, and adding the output image selected from the search result list to a user stamp library of the instant messaging client as a stamp for the instant messaging client in response to a user add operation; 13. An image retrieval method comprising:

2. The step of presetting a plurality of image similarity dimensions according to the similarity level between the two chat dialogue images includes: pre-acquiring a plurality of chat dialogue images for use in the instant messaging client; and The image search method according to claim 1, further comprising a step of hierarchizing information contained in the plurality of chat dialogue images and setting the plurality of image similarity dimensions to any one of a character content dimension, a semantic dimension, a role dimension, a movement dimension, and a conversation relationship dimension.

3. The step of searching and obtaining at least one output image similar to the input image in the plurality of image similarity dimensions includes: For each image similarity dimension, A process of extracting first feature information in the image similarity dimension of the input image; obtaining second feature information for each candidate image in the image database in the image similarity dimension; A process of calculating a similarity between the first feature information and each of the second feature information; and determining, as the output image, a candidate image in the image database that is similar to the input image in the image similarity dimension in response to each calculated similarity measure; The image retrieval method of claim 1 , further comprising the step of:

4. The image retrieval method according to claim 3 , wherein the feature information includes text-like feature information and / or vector-like feature information.

5. generating and displaying a search result list in response to the at least one output image; extracting third feature information of each output image; and The image search method according to claim 1 , further comprising the steps of: sorting each output image according to the third feature information; and obtaining the search result list according to a sorting result.

6. moreover, The image search method according to any one of claims 1 to 5, further comprising a step of sending the target output image to a target chat partner in the chat conversation when a user's confirmation operation on a target output image in the search result list is detected.

7. An image retrieval device, comprising: a first display unit, a determination unit, a setting unit, a retrieval unit, a generation unit, and a second display unit; The first display unit is used for displaying a chat dialogue image received in a chat conversation by an instant messaging client; The determining unit is used for determining the chat dialogue image as an input image for searching; The setting unit is used for presetting a plurality of image similarity dimensions according to a similarity level between two chat dialogue images; The searching unit is used for searching and obtaining at least one output image similar to the input image in the plurality of image similarity dimensions, each output image being similar to the input image in at least one image similarity dimension; the generating unit is adapted to generate a search result list according to the at least one output image; The second display unit is used to display the search result list and, in response to a user's add operation, add the output image selected from the search result list to a user stamp library of the instant messaging client as a stamp of the instant messaging client.

8. The image search device of claim 7, wherein the setting unit is used to pre-acquire a plurality of chat dialogue images used by the instant messaging client, hierarchically organize information contained in the plurality of chat dialogue images, and set the plurality of image similarity dimensions to any one of a character content dimension, a semantic dimension, a role dimension, a movement dimension, and a conversation relationship dimension.

9. 8. The image search device of claim 7, wherein the search unit is used to execute, for each image similarity dimension, a process of extracting first feature information of the input image in that image similarity dimension, a process of acquiring second feature information of each candidate image in the image database in that image similarity dimension, a process of calculating similarities between the first feature information and each second feature information, and a process of determining, in the image database, a candidate image that is similar to the input image in that image similarity dimension as the output image in accordance with each calculated similarity.

10. The image search apparatus according to claim 9 , wherein the feature information includes text-like feature information and / or vector-like feature information.

11. The generating unit further comprises: extracting third feature information of each output image; sorting each output image according to the third feature information, and obtaining the search result list according to a sorting result; The image search device according to claim 7, which is used to perform the above-mentioned.

12. The first display unit further includes: An image search device as described in any one of claims 7 to 11, which is used to send the target output image to a target chat partner in the chat conversation when a user's confirmation operation on a target output image in the search result list is detected.

13. A computer program product for causing a processor to execute the image retrieval method according to any one of claims 1 to 6.

Images