Image search method and device, terminal, storage medium and product

Abstract

The invention provides an image searching method and device, a terminal, a storage medium and a product, and belongs to the technical field of image processing. The method comprises the following steps: for any image in an image set, determining a label of the image; based on the label of the image, a first search map corresponding to the label is determined from a full-amount search map, the first search map comprises the label and multiple pieces of extension information corresponding to the label, and the full-amount search map comprises first search maps corresponding to multiple preset labels; based on the image, performing adaptation configuration on the first search map to obtain a second search map; and when search information is received, determining a first image matched with the search information from the image set based on the search information and the second search map of the images in the image set. The image search is performed based on the second search map, so that the image search accuracy can be improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image search method, device, terminal, storage medium and product. Background Art

[0002] At present, the image shooting function of terminals such as mobile phones is becoming more and more powerful, so users often use terminals to shoot images, resulting in more and more images stored in the terminal's photo album, and then the time consumed by users to search for images from the photo album is increasing. In order to improve the efficiency of users searching for images, an image search function is applied to the terminal's photo album, so that users can search for images from the terminal's photo album through the image search function.

[0003] In the related art, each image in the album is classified by a label classification algorithm to obtain a label for each image. When a user searches for an image, the search keyword input by the user is obtained, and images whose labels meet the matching conditions with the search keyword are obtained from the album.

[0004] Since the labels provided by the label classification algorithm are limited, and the search keywords input by users are diverse, when performing image search based on search keywords and labels, the images searched out may not be the images that users want, resulting in low accuracy of image search. Summary of the invention

[0005] The embodiments of the present application provide an image search method, device, terminal, storage medium and product, which can improve the accuracy of image search. The technical solution is as follows:

[0006] In one aspect, an image search method is provided, the method comprising:

[0007] For any image in the image set, determining a label of the image;

[0008] Based on the label of the image, determining a first search graph corresponding to the label from the full search graph, wherein the first search graph includes the label and a plurality of extended information corresponding to the label, and the full search graph includes a first search graph corresponding to a plurality of preset labels;

[0009] Based on the image, adaptively configuring the first search graph to obtain a second search graph;

[0010] When the search information is received, a first image matching the search information is determined from the image set based on the search information and a second search graph of images in the image set.

[0011] In another aspect, an image search device is provided, the device comprising:

[0012] A first determination module, configured to determine a label of any image in the image set;

[0013] A second determination module is used to determine, based on the label of the image, a first search map corresponding to the label from the full search map, wherein the first search map includes the label and a plurality of extended information corresponding to the label, and the full search map includes a first search map corresponding to a plurality of preset labels;

[0014] A configuration module, configured to adaptively configure the first search graph based on the image to obtain a second search graph;

[0015] The search module is used to determine a first image matching the search information from the image set based on the search information and a second search graph of images in the image set when the search information is received.

[0016] On the other hand, a terminal is provided, which includes one or more processors and one or more memories, wherein at least one program code is stored in the one or more memories, and the at least one program code is loaded and executed by the one or more processors to implement the image search method described above.

[0017] On the other hand, a computer-readable storage medium is provided, wherein at least one program code is stored in the storage medium, and the at least one program code is loaded and executed by a processor to implement the above-mentioned image search method.

[0018] On the other hand, a computer program product is provided. The computer program product stores at least one program code. The at least one program code is used to be executed by a processor to implement the above-mentioned image search method.

[0019] In the embodiment of the present application, on the one hand, the first search graph includes the label of the image and multiple extended information corresponding to the label, and the multiple extended information can enrich the information content of the image search. On the other hand, the second search graph is obtained by adaptively configuring the first search graph based on the image; therefore, the present application can establish a personalized second search graph. Based on the above discussion, it can be seen that performing image search based on the second search graph can improve the accuracy of image search. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 A schematic diagram showing an implementation environment of an image search method shown in an exemplary embodiment of the present application;

[0021] Figure 2 A flowchart of an image search method shown in an exemplary embodiment of the present application is shown;

[0022] Figure 3 A modular block diagram of an image search shown in an exemplary embodiment of the present application is shown;

[0023] Figure 4 A schematic diagram of generating a second search tree according to an exemplary embodiment of the present application is shown;

[0024] Figure 5 A flowchart of an image search method shown in an exemplary embodiment of the present application is shown;

[0025] Figure 6 A schematic diagram of generating extended information is shown in an exemplary embodiment of the present application;

[0026] Figure 7 A flowchart of an image search method shown in an exemplary embodiment of the present application is shown;

[0027] Figure 8 An image search interface diagram shown in an exemplary embodiment of the present application is shown;

[0028] Fig. 9 An image search interface diagram shown in an exemplary embodiment of the present application is shown;

[0029] Fig.10 A flowchart of an image search method shown in an exemplary embodiment of the present application is shown;

[0030] Fig.11 A flowchart of an image search method shown in an exemplary embodiment of the present application is shown;

[0031] Fig.12 A schematic diagram of generating an image information library according to an exemplary embodiment of the present application is shown;

[0032] Fig.13 A flowchart of an image search method shown in an exemplary embodiment of the present application is shown;

[0033] Fig.14 A flowchart of an image search method shown in an exemplary embodiment of the present application is shown;

[0034] Fig.15 A block diagram of an image search device shown in an exemplary embodiment of the present application is shown;

[0035] Fig.16 A block diagram of a terminal according to an exemplary embodiment of the present application is shown. DETAILED DESCRIPTION

[0036] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0037] The term "multiple" as used herein refers to two or more than two. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the related objects are in an "or" relationship.

[0038] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.) and signals involved in this application are all authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions. For example, the control messages and data messages involved in this application are all obtained with full authorization.

[0039] Please refer to Figure 1 , which shows a schematic diagram of an implementation environment of an image search method shown in an exemplary embodiment of the present application. Figure 1 The implementation environment includes: a terminal 101 and a server 102. An image processing application is installed in the terminal 101, and the terminal 101 implements functions such as data transmission and information interaction with the server 102 through the image processing application. For example, the terminal 101 interacts with the server 102 through the image processing application to implement an image search function. The image processing application can be a photo album, and the server 102 can be a background server 102 corresponding to the photo album.

[0040] In some embodiments, the terminal 101 may be a mobile phone, a tablet computer, or a laptop computer. The server 102 may be an independent physical server 102, or a server 102 cluster or a distributed system composed of multiple physical servers 102, or a cloud server 102 that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms.

[0041] Please refer to Figure 2 , which shows a flowchart of an image search method shown in an exemplary embodiment of the present application. Figure 2 , the method comprising:

[0042] Step 201: For any image in the image set, determine the label of the image.

[0043] The image collection is the photo album of the terminal, and the sources of the images in the photo album include the photos taken by this terminal, the photos taken by other terminals, and the application software in the terminal. Other terminals can be terminals of different types from this terminal or terminals of the same type as this terminal, and the application software from the terminal can be social software, shopping software, or browsers from the terminal. In some embodiments, the image collection includes multiple images, and the multiple images may be stored in the image collection at different times. In this step, whenever an image is added to the image collection, the terminal determines the label of the newly added image, thereby improving the timeliness of determining the label of the image. In other embodiments, the terminal can periodically determine the newly added images in the image collection and determine the labels of the newly added images, that is, periodically determine the labels of the newly added graphics in the recent period at one time, thereby improving processing efficiency. For example, the terminal determines the newly added images in the image collection of this day during the idle time of each day (for example, a fixed time every night) and determines the labels of the newly added images, thereby avoiding the situation where the label of the image is determined, the processor of the terminal is occupied, and the terminal responds to other operations of the user at a slow speed.

[0044] In some embodiments, the terminal stores an image classification model; the terminal determines the label of the image through the image classification model; accordingly, the terminal inputs the image into the image classification model to obtain the probability that the image belongs to multiple labels, and based on the probability that the image belongs to multiple labels, the label with the highest probability is used as the label of the image. Since the image classification model is stored in the terminal, the terminal can perform image classification offline, that is, even if the terminal is offline, the terminal can also determine the label of the image. In other embodiments, the terminal does not store the image classification model, but stores the image classification model in the server; accordingly, the step of determining the label of the image by the terminal can be: the terminal sends the image to the server, the server receives the image, determines the probability that the image belongs to multiple labels through the image classification model, and returns the probability that the image belongs to multiple labels to the terminal; the terminal receives the probability that the image belongs to multiple labels, and based on the probability that the terminal belongs to multiple labels, the label with the highest probability is used as the label of the image. Since the image classification model is not stored in the terminal and the terminal does not perform image classification in person, the storage resources and computing resources of the terminal can be saved.

[0045] For example, the image set includes 1.jpg and 2.jpg. The terminal determines that the probability that 1.jpg belongs to the label "person" is 0.88, the probability that it belongs to the label "dog" is 0.66, and the probability that it belongs to the label "grass" is 0.99. Since the probability that 1.jpg belongs to the label "grass" is the highest, the terminal determines that the label of 1.jpg is the label "grass". The terminal determines that the probability that 2.jpg belongs to the label "office" is 0.77, and the probability that it belongs to the label "person" is 0.99. Since the probability that 2.jpg belongs to the label "person" is the highest, the terminal determines that the label of 2.jpg is the label "person".

[0046] In some embodiments, see Figure 3 The terminal includes an information collection module, which is used to obtain labels of images in the image set through a label classification algorithm. In other embodiments, the information collection module can also obtain text information in the images in the image set through an optical character recognition (OCR) algorithm, and can also obtain attribute information of the images; for example, obtaining the shooting time and shooting location of the images.

[0047] Step 202: Based on the label of the image, determine a first search graph corresponding to the label from the full search graph, the first search graph includes the label and multiple extended information corresponding to the label, and the full search graph includes first search graphs corresponding to multiple preset labels.

[0048] The multiple extended information corresponding to the tag includes multiple synonyms, hypernyms, hyponyms, associative words, correction words, completion words, recommended words and custom words of the tag. In an embodiment of the present application, the multiple extended information corresponding to the tag includes synonyms, hypernyms, hyponyms, associative words, correction words, completion words, recommended words and custom words of the tag as an example for illustration, thereby increasing the amount of extended information. The more extended information there is, the more accurate the images recommended to the user are. Therefore, the present application can improve the accuracy of image search.

[0049] The first search graph can be in the form of a table or a search tree; when the first search graph is in the form of a search tree, the root node of the first search tree is the label of the image, and the leaf nodes of the first search tree are the multiple extended information corresponding to the label. In addition, the priorities of the multiple extended information can be set, the root node of the first search tree is located on the left, and based on the priorities of the multiple extended information, the leaf nodes are arranged from top to bottom, so that the extended information with a high priority is arranged at the top, and the extended information with a low priority is arranged at the bottom.

[0050] In some embodiments, the terminal stores the full search graph locally. In this step, the terminal obtains the first search graph corresponding to the tag from the locally stored full search graph based on the tag of the image. Since the full search graph is stored in the terminal, the terminal can obtain the first search graph corresponding to the tag from the full search graph offline, and then perform image search offline. In other embodiments, the terminal does not store the full search graph locally, and the terminal determines the first search graph corresponding to the tag with the help of the server; accordingly, step 202 can be: the terminal sends a request to the server to obtain the tag of the image; the server receives the request from the terminal, obtains the first search graph corresponding to the tag from the full search graph based on the tag of the image, and sends the first search graph corresponding to the tag to the terminal; the terminal receives the first search graph corresponding to the tag sent by the server. Since the terminal does not need to store the full search graph locally, the storage resources and computing resources of the terminal are saved.

[0051] In the embodiment of the present application, the full search graph includes the first search graphs corresponding to multiple preset tags, that is, the full search graph provides a rich and complete search graph, and the terminal only needs to obtain the first search graph corresponding to the tag according to the image tag, without obtaining the first search graphs corresponding to other tags, thereby streamlining a large number of search keyword recommendations and improving search efficiency and user experience. Figure 4 The full search graph includes the first search graph corresponding to "dog" and the first search graph corresponding to "cat"; and there are images with the label "dog" in the image collection, but no images with the label "cat"; therefore, the terminal only needs to obtain the first search graph corresponding to "dog", and there is no need to obtain the first search graph corresponding to "cat".

[0052] Step 203: Based on the image, adaptively configure the first search graph to obtain a second search graph.

[0053] The first search graph includes multiple extended information, and some of the multiple extended information may not be suitable for the image. Therefore, in this step, based on the image, the extended information in the first search graph is adaptively configured to establish a set of personalized second search graphs to provide users with a personalized search experience. This step can be implemented by the following steps (1) to (3), including:

[0054] (1) For any extended information in the first search graph, determine whether the image is newly associated with the extension.

[0055] The multiple extended information corresponding to the tag include synonyms, hypernyms, hyponyms, associative words, error correction words, completion words, recommended words and custom words of the tag. In the case where the extended information is a synonym of the tag, or in the case where the extended information is a hypernym of the tag, or in the case where the extended information is a hyponym of the tag and the image includes image content corresponding to the hyponym, or in the case where the extended information is an error correction word of the tag, or in the case where the extended information is a completion word of the tag, or in the case where the extended information is a recommended word of the tag and the image includes image content corresponding to the recommended word, or in the case where the extended information is a custom word of the tag, it is determined that the image is associated with the extended information. In the case where the extended information is a hyponym of the tag, but the image does not include image content corresponding to the hyponym, or in the case where the extended information is a recommended word of the tag, but the image does not include image content corresponding to the recommended word, it is determined that the image is not associated with the extended information. In some embodiments, when the image is associated with the extended information, step (2) is performed; and when the image is not associated with the extended information, step (3) is performed.

[0056] (2) When the image is associated with the extended information, the extended information is determined as a search object in the first search graph.

[0057] The state of the extended information is set to a valid state in the first search graph, and the extended information in a valid state can be used as a search object for image search. For example, if the initial state of multiple extended information in the first search graph is an invalid state, the extended information is lit up in the first search graph to set the state of the extended information to a valid state. For another example, if the initial state of multiple extended information in the first search graph is a valid state, the state of the extended information in the first search graph is maintained, that is, the lighted state of the extended information is maintained.

[0058] (3) When the image is not associated with the extended information, the extended information is determined as a non-search object in the first search graph to obtain a second search graph.

[0059] The state of the extended information is set to an invalid state in the first search graph, and the extended information in the invalid state cannot be used as a search object for image search. For example, if the initial state of multiple extended information in the first search graph is an invalid state, the state of the extended information is maintained in the first search graph. For another example, if the initial state of multiple extended information in the first search graph is a valid state, the extended information is extinguished in the first search graph to set the state of the extended information to an invalid state.

[0060] In some embodiments, the terminal can directly determine the second search map by lighting up the full search map; for example, continue to refer to Figure 4, the first search graph corresponding to "dog" includes multiple extended information including "dog, dog, dog", "canine", "pet", "mammal", "pug, golden retriever, Chihuahua", "grass, child", "cat", "sit", etc. Assume that there are images with the label "dog" in the image collection, there are no images with the label "cat" in the image collection, and the images of "dog" only include pug, people, and grass. Then the terminal lights up "dog, dog, dog", "canine", "pet", "mammal", "pug", "grass, child", "sit", etc., while other contents in the full search graph continue to remain off. In some embodiments, Figure 4 In the description, the second search graph is a search tree as an example; when the second search graph is in the form of a table, the second search graph is Table 1:

[0061] Table 1

[0062]

[0063] Based on Table 1 above, it can be seen that the second search graph will inherit the label, synonyms / near-synonyms, error correction words, hypernyms, etc. of "dog", but for hyponyms, since there is only pug in the dog category, the hyponym of dog is only pug, and the completion word will not complete the phrases of other dogs, and the recommended category will not recommend searches related to cats, thereby simplifying the recommended scope of the search, reducing redundant information, and improving the accuracy of the search, thereby improving the user experience.

[0064] In some embodiments, continue to refer to Figure 3 The terminal also includes a search system module, which is used to establish a search system based on the image collection by lighting up the search tree, and construct hypernyms, hyponyms, synonyms / synonyms, correction words, associative words, recommended words, completion words and custom words for tags to better match user searches.

[0065] In an embodiment of the present application, the first search graph corresponding to the label provided by the full search graph includes multiple extended information. When establishing the second search graph in this step, it is only necessary to light up the local material, that is, the image-related extended information included in the image collection, and other parts do not need to be lit up, thereby simplifying the second search graph and allowing users to obtain a personalized search graph, reducing unnecessary search actions, and improving search efficiency and user experience.

[0066] In some embodiments, the image set includes multiple images. For each image, the second search graph corresponding to each image is determined according to the above steps 201-203, and then step 204 is executed.

[0067] Step 204: When the search information is received, based on the search information and a second search graph of images in the image set, determine a first image matching the search information from the image set.

[0068] The search information may be a first search keyword or a reference image; when the search information is the first search keyword, for any image in the image set, the matching degree between the first search keyword and the label and extended information included in the second search graph of the image is determined; based on the matching degree, a first image whose matching degree satisfies the condition is determined from the image set. For example, based on the matching degree, multiple first images with the highest matching degree are determined from the image set; or based on the matching degree, multiple first images with matching degrees exceeding a preset threshold are determined from the image set. Since the number of image labels is generally limited, and the user search methods are varied, it is difficult for the algorithm to cover all the search labels that the user wants to use, which makes it difficult for the user to search for the image that the user wants, resulting in low search accuracy. In the embodiment of the present application, a first search graph is established according to the label of the image, and the first search graph is personalized based on the image. When the user enters the search keyword, the terminal will find the corresponding image according to the extended information or label matched from the first search graph, thereby improving the accuracy of the image search.

[0069] In the case where the search information is a reference image, a second search keyword is determined based on the reference image, and a first image matching the second search keyword is determined from the image set based on the second search keyword and a second search graph of images in the image set. This process is similar to the process of determining a first image matching the first search keyword from the image set based on the first search keyword and a second search graph of images in the image set, and will not be described in detail here. In the embodiment of the present application, not only can image search be performed by search keywords, but also the ability to search images by image is added, so that users can find some familiar images based on reference images, thereby enriching the image search function.

[0070] In some embodiments, continue to refer to Figure 3 The terminal also includes a search interaction module, which obtains search information input by the user. The search information can be a first search keyword or a reference image. When the user inputs the first search keyword, the search interaction module recommends an image to the user based on the first search keyword. When the user inputs a reference image, the search interaction module recommends an image to the user based on the reference image input by the user, thereby realizing image search and obtaining similar images.

[0071] In the embodiment of the present application, on the one hand, the first search graph includes the label of the image and multiple extended information corresponding to the label, and the multiple extended information can enrich the information content of the image search. On the other hand, the second search graph is obtained by adaptively configuring the first search graph based on the image; therefore, the present application can establish a personalized second search graph. Based on the above discussion, it can be seen that performing image search based on the second search graph can improve the accuracy of image search.

[0072] Please refer to Figure 5 , which shows a flowchart of an image search method shown in an exemplary embodiment of the present application. Figure 5 , the method comprising:

[0073] Step 501: For any preset tag among a plurality of preset tags, the terminal expands the preset tag based on a plurality of extension words to obtain a plurality of extension information corresponding to the preset tag.

[0074] The multiple preset tags are multiple tags of the image classification model, that is, the embodiment of the present application utilizes the characteristics of the cloud-based large model to establish a full search graph based on the multiple preset tags provided by the image classification algorithm. The multiple extended words include multiple synonyms, hypernyms, hyponyms, associative words, error correction words, completion words, recommended words, and custom words; accordingly, the terminal expands the preset tags based on the multiple extended words to obtain multiple extended information corresponding to the preset tags can be implemented by the following steps (1) to (2), including:

[0075] (1) The terminal determines multiple words among the synonyms, hypernyms, hyponyms, associative words, completion words, recommended words and custom words of the preset tag.

[0076] Synonyms: Since the labels provided by the image classification algorithm can only have certain specific words, and each user may have different cognitions of different categories, synonyms can help users search; for example, the synonyms of "baby" can be "baby", "infant", "little baby" and "little baby". Hypernym: The upper-level relationship corresponding to this preset label; for example, the hypernym of "dog" can be "animal", "pet" and "mammal". For another example, the hypernym of "maple tree" can be "tree" and "plant". Hyponym: Some sub-category labels corresponding to this preset label; for example, the hyponyms of "dog" are "golden retriever" and "pug". Completion word: When users search for images, some words are incomplete and need to be completed. For example, the completion words of "card" can be "bank card", "work card" and "bus card". Associative word: When users search for images, they do not fully know which images they really want, and need some associative words to assist users in association. The main scope of association comes from the subject label in the same image. For example, if a user wants to search for images of himself and his dog playing together, the user can recommend keywords such as "person" and "grass" when searching for "dog". Error correction: Correct some frequently used words, such as those that are misspelled in the wrong order, etc. For example, "machine doll" is changed to "Chihuahua".

[0077] Recommended words: When a user enters a search keyword to search an image, there may be a situation where there are no search keywords entered by the user. At this time, some keywords that the user may want to search can be recommended to the user; for example, the user wants to search for Van Gogh, and the search keyword Van Gogh is not within the preset tag range of the algorithm. At this time, keywords such as "picture" and "oil painting" can be recommended to the user. Accordingly, the step of the terminal determining the recommended words of the preset tag can be: the terminal determines the type of the preset tag, and determines the type as the recommended word of the preset tag. Custom words: Since different preset tags have different names in different regions, especially many dialects, users can define their own names based on pictures to supplement the search keywords. Accordingly, the step of the terminal determining the custom words of the preset tag can be: the terminal determines multiple dialect names of the preset tag, and determines the multiple dialect names of the preset tag as the custom words of the preset tag.

[0078] In some embodiments, the above synonyms, hypernyms, hyponyms, associative words, completion words, recommended words and custom words and the methods of determining synonyms, hypernyms, hyponyms, associative words, completion words, recommended words and custom words are merely examples, and other words may also be expanded and the expanded information may be determined by other methods.

[0079] (2) The terminal combines multiple synonyms, hypernyms, hyponyms, associative words, completion words, recommended words and custom words of the preset tag into multiple extended information.

[0080] Multiple extended information can be used as search objects for image search; for example, please refer to Figure 6 , the preset label is label 1, the synonym of label 1 is used as search object 1, the correction word of label 1 is used as search object 2, the completion word of label 1 is used as search object 3, the custom word of label 1 is used as search object 4, the hypernym of label 1 is used as search object 5, the hyponym of label 1 is used as search object 6, the associative word of label 1 is used as search object 7, and the recommended word of label 1 is used as search object 8.

[0081] In some embodiments, the terminal can also determine the priority level of a preset tag and the priority level of multiple extended words, and store the priority level of the preset tag and the priority level of multiple extended words, so that when the user subsequently performs an image search, the recommendation index is recommended based on the priority level of the tag and the priority level of multiple extended words. Among them, the terminal can set a priority field for the preset tag and the multiple extended words respectively, and the priority field is used to store the priority level value, and the lower the priority level value, the higher the priority level. The priority level value of the preset tag is 1, the priority level value of the synonym is 2, the priority level value of the complement word is 3, the priority level value of the error correction word is 4, and the priority levels of the hypernym, hyponym, associative word, recommended word and custom word are the same, and their priority levels are all 5.

[0082] Step 502: The terminal generates a first search graph corresponding to the preset tag based on the preset tag and multiple extended information corresponding to the preset tag.

[0083] The first search graph includes a preset tag and multiple extended information corresponding to the preset tag; and the first search graph can be in the form of a table or a search tree; for example, the first search graph is in the form of a search tree, and the preset tags include "dog" and "cat", then the terminal generates the first search graph corresponding to "dog" and "cat" as follows Figure 4 The two tags in the left figure and the extended information corresponding to the tags. For another example, the first search graph is in the form of a table, and the preset tags include "dog" and "cat"; the terminal generates the first search graph corresponding to "dog" and "cat" as shown in Table 2 below:

[0084] Table 2

[0085]

[0086] We set up a search graph to aggregate a large number of keywords from different search methods according to the tags that the algorithm can provide, thereby improving the effectiveness of search input, simplifying the tags that are constantly increasing on the algorithm side, and reducing the cost of algorithm development. Then, based on the user's localized image materials, we further streamline the search graph by lighting up the search tree, making the user's search graph more personalized, reducing unnecessary recommendation redundancy, and allowing users to quickly and efficiently find the images they need.

[0087] Step 503: The terminal combines the first search graphs corresponding to the plurality of preset tags into a full search graph.

[0088] In some embodiments, the server may generate a full search graph, and the terminal may obtain the generated full search graph from the server, thereby saving computing resources of the terminal. The method for the server to generate a full search graph is the same as the method for the terminal to generate a full search graph, which will not be repeated here.

[0089] In an embodiment of the present application, a full search graph is established based on multiple preset tags provided by the image classification algorithm by utilizing the characteristics of the cloud-based large model. The multiple preset tags provided by the image classification algorithm are fixed; therefore, by establishing a full search graph based on multiple preset tags in advance, a rich and complete graph can be established. Subsequently, the corresponding first search graph can be found from the full search graph directly based on the tags, and the first search graph can be personalized to obtain a personalized second search graph, which provides efficiency in determining the second search graph and thus improves the efficiency of subsequent image searches. By constructing a search graph, the effectiveness of keywords entered by users is improved, the number of tags that the algorithm needs to output is reduced, the difficulty of algorithm training is reduced, and the availability of search keywords is improved from a logical level.

[0090] Please refer to Figure 7 , which shows a flowchart of an image search method shown in an exemplary embodiment of the present application. Figure 7 , the method comprising:

[0091] Step 701: For any image in the image set, the terminal determines a label of the image.

[0092] In some embodiments, this step is the same as step 201 and will not be repeated here.

[0093] Step 702: Based on the label of the image, the terminal determines a first search graph corresponding to the label from the full search graph, the first search graph includes the label and multiple extended information corresponding to the label, and the full search graph includes the first search graph corresponding to multiple preset labels.

[0094] In some embodiments, this step is the same as step 202 and will not be repeated here.

[0095] Step 703: The terminal performs adaptability configuration on the first search graph based on the image to obtain a second search graph.

[0096] In some embodiments, this step is the same as step 203 and will not be elaborated here.

[0097] Step 704: When receiving a first search keyword, the terminal determines a recommended index based on the first search keyword and the second search graph of the images in the image set.

[0098] For the second search graph of any image in the image set, the terminal determines the matching degree between the first search keyword and the labels and multiple extended information in the second search graph; when the matching degree between the first search keyword and the labels in the second search graph exceeds a preset threshold, the label is used as the recommended index; when the matching degree between the first search keyword and any extended information in the second search graph exceeds the preset threshold, the extended information is used as the recommended index. In some embodiments, the terminal can simultaneously determine the matching degree between the first search keyword and the labels and multiple extended information in the second search graph, so as to determine as many recommended indexes for the user as possible. The terminal can also first determine the matching degree between the first search keyword and the labels in the second search graph. When the matching degree exceeds the preset threshold, the label is determined as the recommended index, and at this time, the matching degree between the first search keyword and the multiple extended information in the second search graph is no longer determined, thereby reducing the calculation amount and improving the efficiency of determining the recommended index. When the matching degree does not exceed the preset threshold, the matching degree between the first search keyword and the multiple extended information in the second search graph is determined, and then based on the matching degree, a recommended index is determined for the user from the multiple extended information.

[0099] For example, please refer to Figure 8 , in search interface 1, the first search keyword input by the user is "gou", and the label "gou" does not exist in the second search graph. At this time, it is necessary to correct the word "gou", that is, the recommended index matched by the terminal based on "gou" is the corrected word "dog". Another example is that the first search keyword input by the user is "identity card", and the terminal determines that the recommended index of "identity card" is the corrected word "ID card". Another example, continue to refer to Figure 8In search interface 2, the first search keyword input by the user is "dog", and the second search graph contains the hyponym "pug" of "dog", then the terminal determines "pug" as the recommended index. In an embodiment of the present application, the terminal determines the recommended index for the user based on the second search graph, and the extended information not in the second search graph is not recommended. For example, although "dog" can be associated with "wolf dog", there is no "wolf dog" in China in the second search graph, so "wolf dog" is not used as a recommended index, thereby simplifying image search and improving search efficiency. For another example, continue to refer to Figure 8 In search interface 3, the first search keyword entered by the user is "dog", and the terminal matches the recommended indexes "people" and "grass" based on "dog". For another example, the first search keyword entered by the user is "dog", "dog" or "dog", and the terminal determines the recommended index as "dog" based on the first search keyword.

[0100] Step 705: The terminal displays the recommended index.

[0101] The terminal displays the recommended index, and the user can quickly obtain the first image corresponding to the recommended index by clicking on the recommended index. In some embodiments, when the user clicks on the recommended index, a first search instruction is triggered to the terminal, the first search instruction carries the recommended index, and then step 706 is executed. In some embodiments, when there are multiple recommended indexes, the terminal displays multiple recommended indexes based on the order of the word frequencies of the multiple recommended indexes from large to small. Moreover, when the number of multiple recommended indexes exceeds a preset number, the preset number is the maximum number of search indexes that can be displayed on the current search interface, and the user can slide the recommended index to select. The process may be: the terminal displays a preset number of recommended indexes based on the order of the word frequencies of the multiple recommended indexes from large to small, and when a sliding operation triggered by the user is received, the recommended index displayed in the search interface is replaced based on the sliding operation. For example, please refer to Fig. 9 In the search interface 4, the terminal determines three recommended indexes based on the first search keyword "dog", namely "pug", "wolf dog" and "local dog".

[0102] In some other embodiments, the user may also search directly based on the first search keyword input by the user. In this case, the user may click the search button to trigger a second search instruction to the terminal, and the second search instruction carries the first search keyword. In response to the second search instruction, the terminal determines a first image matching the first search keyword based on the first search keyword and the second search graph of images in the image set. The step of determining the first image matching the first search keyword based on the first search keyword and the second search graph of images in the image set may be: for any image in the image set, determining the degree of matching between the first search keyword and the label and extended information included in the second search graph of the image; based on the degree of matching, determining a first image from the image set whose degree of matching meets the condition. For example, based on the degree of matching, determining multiple first images with the highest degree of matching from the image set; or based on the degree of matching, determining multiple first images from the image set whose degree of matching exceeds a preset threshold.

[0103] Step 706: In response to the recommendation index being triggered, the terminal obtains a first image corresponding to the recommendation index.

[0104] In some embodiments, the recommended index is a tag, and the tag is associated with an image; the terminal then obtains the image associated with the tag; in other embodiments, the recommended index is extended information, the extended information is associated with a tag, and the tag is associated with an image; the terminal then obtains the tag associated with the extended information and obtains the image associated with the tag.

[0105] In an embodiment of the present application, when a user enters a first search keyword, the terminal recommends a recommended index for the first search keyword based on a second search graph of images in the image collection, and the user can then directly click on the recommended index to quickly and accurately obtain the desired image, thereby improving the image search efficiency and accuracy.

[0106] Please refer to Fig.10 , which shows a flowchart of an image search method shown in an exemplary embodiment of the present application. Fig.10 , the method comprising:

[0107] Step 1001: For any image in an image set, the terminal determines a label of the image.

[0108] In some embodiments, this step is the same as step 201 and will not be repeated here.

[0109] Step 1002: Based on the label of the image, the terminal determines a first search graph corresponding to the label from the full search graph, the first search graph includes the label and multiple extended information corresponding to the label, and the full search graph includes the first search graph corresponding to multiple preset labels.

[0110] In some embodiments, this step is the same as step 202 and will not be repeated here.

[0111] Step 1003: Based on the image, the terminal adaptively configures the first search graph to obtain a second search graph.

[0112] In some embodiments, this step is the same as step 203 and will not be repeated here.

[0113] Step 1004: When the image search function is started for the first time, the terminal displays a setting interface, which is used to set index configuration information.

[0114] The index configuration information includes at least one of a label probability threshold, a priority level of multiple extended information, and a first identifier, the first identifier is used to indicate whether to use text information in the image as at least one of the search objects; the label probability threshold is used to constrain the determined label of the image, and the priority level of multiple extended information is used to determine the matching order when recommending the index. The setting interface includes prompt information, a confirmation button, and a negative button. The prompt information is used to prompt whether to use the default index configuration information. If the user wants to use the default index configuration information, the user can trigger the confirmation button. If the user wants to customize the index configuration information, the user can trigger the negative button. For example, please refer to Fig.11 When the user enables search for the first time, a prompt message is displayed to prompt whether to use the default index configuration information.

[0115] Step 1005: The terminal obtains index configuration information based on the settings in the settings interface and stores the index configuration information, where the index configuration information is the default index configuration information or the index configuration information customized by the current login account.

[0116] In response to the confirmation button being triggered, the terminal obtains the default index configuration information and stores the default index configuration information. In response to the negative button being triggered, the priority setting interface is displayed. The priority setting interface is used to set the priority of the label and multiple extended information in the second search graph. The multiple extended information is obtained by expanding the label based on multiple extended words; therefore, the priority of the multiple extended information is the priority of the multiple extended words; wherein the priority setting interface includes the label and its corresponding priority value and multiple extended words and the priority value corresponding to each extended word. Moreover, the priority value corresponding to the label and the priority value corresponding to each extended word are editable, and the user can customize the priority of the label and the priority of multiple extended words by editing the priority value; the terminal obtains the label and the priority value of multiple extended words set based on the priority interface, and then displays the label probability threshold setting interface. The user can set the label probability threshold in the label probability threshold setting interface. The terminal obtains the label probability threshold set based on the label probability threshold setting interface, and then displays the text search setting interface. The text search setting interface is used to set the first identifier, and the first identifier is used to indicate whether to use the text information in the image as the search object. For example, continue to refer to Fig.11 , user-defined priority levels of multiple extended words, customized label probability threshold, and customized whether to use text information in the image as the search object.

[0117] In some embodiments, when a user starts an image search for the first time, the user needs to select a search rule once, that is, the terminal stores the index configuration information that the user has set, so that when the user uses the image search again, there is no need to select the search rule again. In addition, the user can also modify the set index configuration information later, and the process can be: in response to the setting option being triggered, the setting interface is displayed, and the index configuration information modified based on the setting interface is obtained. In an embodiment of the present application, by setting the search priority, the user can obtain a better recommendation order. In addition, through user customization of the priority level and label probability threshold, the user can obtain a further search experience.

[0118] Step 1006: When the first search keyword is received, the terminal obtains the stored index configuration information, and determines a recommended index from a second search graph of images in the image set based on the search configuration information and the first search keyword.

[0119] The index configuration information includes at least one of a label probability threshold, a priority level of multiple extended information, and a first identifier, and the first identifier is used to indicate whether to use text information in the image as a search object. Accordingly, the terminal determines a recommended index from a second search graph of images in the image set based on the search configuration information and the first search keyword, including at least one of the following implementation methods:

[0120] The first implementation method: for any label of an image in the image set, if the probability of the label is less than the label probability threshold, the terminal deletes the label, obtains a third search graph of any image in the image set, and determines a recommended index from the third search graph of images in the image set based on the first search keyword. In an embodiment of the present application, the label probability threshold set by the user can help the user further filter out some low-probability labels, further simplify the second search graph, and thus improve the efficiency of image search based on the second search graph.

[0121] In some embodiments, please refer to Fig.12 , the terminal stores an image information library of each image in the image set, and the image information library of the image includes the label to which the image belongs and the probability of the label to which the image belongs determined based on the image classification algorithm; therefore, in this step, the terminal determines the probability of the image from the image information library of the image, and then performs a label filtering operation.

[0122] The second implementation method: the terminal performs matching operations on the tags and the multiple extended information with the first search keyword in turn based on the priority levels of the tags and the priority levels of the multiple extended information included in the second search graph of the images in the image set to obtain a recommended index. In an embodiment of the present application, the recommended index is matched based on the priority levels of the multiple tags and the priority levels of the multiple extended information, and the recommendation of the recommended index can be personalized.

[0123] A third implementation method: When the first identifier is used to indicate that text information in an image is used as a search object, the terminal determines a recommended index from the text information of the image in the image set and the second search map based on the first search keyword. When the first identifier is used to indicate that text information in an image is not used as a search object, the terminal determines a recommended index from the second search map of the image in the image set based on the first search keyword. Since the text information of the image is combined when determining the recommended index, the search object is further enriched, thereby improving the accuracy of the image search.

[0124] In some embodiments, the terminal determines a recommended index from the text information of an image in an image set and a second search graph based on a first search keyword, including: the terminal determines the attribute information of an image in an image set; based on the first search keyword, the recommended index is determined from the text information, attribute information, and second search graph of the image in the image set. The attribute information includes the source of the image, the shooting time of the image, and the shooting location of the image. In an embodiment of the present application, since the recommended index is determined by combining not only the text information of the image but also the attribute information of the image, the search object is further enriched, so the accuracy of the image search can be further improved. And since the attribute information of the image is combined when determining the recommended index, the attribute information includes the shooting time, shooting location, and source of the image, the present application also supports users to search for images by time, location, and source.

[0125] In some embodiments, continue to refer to Fig.12 , the terminal stores an image information library of the image, the image information library also includes text information and attribute information in the image, the text information is determined based on the OCR algorithm, and the attribute information includes the image source, the shooting location of the image, and the shooting time of the image. Accordingly, in this step, the terminal obtains the text information and attribute information of the image from the image information library of the image, and then recommends the recommended index.

[0126] In some embodiments, for any image in the image set, the process of the terminal generating an image information library of the image may be: the terminal determines the label to which the image belongs and the probability that the image belongs to each label based on the image classification algorithm; the terminal obtains the text information in the image based on the OCR algorithm; when the image does not contain text information, the text information is not extracted; the terminal reads the location information of the image, which may be the Global Positioning System (GPS) location information or location information; the terminal reads the time information of the image; the terminal reads the source information of the image. The terminal generates an image information library of the image based on the above acquired information. For example, the terminal generates an image information library of two images as shown in Table 3 below.

[0127] Table 3

[0128]

[0129] Image name: read the actual name of the image in the terminal and save it in this column. The value cannot be empty; label multi-value column: the label of the image predicted by the image classification algorithm and the probability of the label; if the image contains people (0.88), dogs (0.66), grass (0.99), etc., the value can be empty; time information: the time information when the photo was taken, the value can be empty; location information: the location information where the photo was taken (can be GPS coordinates or specific address information), the value can be empty; source information: the device used to capture the image or the application used for transmission (device type: a certain camera, a certain type of mobile phone, a certain smart device, etc.; application type: a certain social application, a certain short video application, a certain check-in application, etc.), the value can be empty; text information: the text information in the image extracted by the OCR algorithm, the value can be empty.

[0130] Step 1007: In response to the recommendation index being triggered, the terminal obtains a first image corresponding to the recommendation index.

[0131] In some embodiments, this step is the same as step 706 and will not be repeated here.

[0132] In an embodiment of the present application, a user can customize index configuration information, so that the terminal determines a recommended index for the user based on the user-defined index configuration information, which can improve the accuracy of the determined recommended index and thereby improve the accuracy of image search.

[0133] Please refer to Fig.13 , which shows a flowchart of an image search method shown in an exemplary embodiment of the present application. Fig.13 , the method comprising:

[0134] Step 1301: For any image in the image set, the terminal determines a label of the image.

[0135] In some embodiments, this step is the same as step 201 and will not be repeated here.

[0136] Step 1302: Based on the label of the image, the terminal determines a first search graph corresponding to the label from the full search graph, the first search graph includes the label and multiple extended information corresponding to the label, and the full search graph includes the first search graph corresponding to multiple preset labels.

[0137] In some embodiments, this step is the same as step 202 and will not be repeated here.

[0138] Step 1303: Based on the image, the terminal adaptively configures the first search graph to obtain a second search graph.

[0139] In some embodiments, this step is the same as step 203 and will not be repeated here.

[0140] Step 1304: When receiving the reference image, the terminal determines a second search keyword based on the reference image.

[0141] The reference image can be an image selected from the image collection or a currently captured image; wherein the image search interface includes a search button, and in response to the search button being triggered, an image selection interface and a capture button are displayed; when the user wants to select a reference image from the image selection interface, the terminal obtains the image selected in the image selection interface and uses the selected image as the reference image. When the user wants to capture an image, the user can click the capture button, the corresponding capture button is triggered, the capture interface is displayed, the image captured in the capture interface is obtained, and the currently captured image is used as the reference image. For example, continue to refer to Fig. 9 , users can also search for the desired images by searching by image or taking a photo; the search interface is as follows Fig. 9 In the search interface 5, a reference image can be input or captured on the right side of the search box in the search interface 5.

[0142] In some embodiments, the terminal determines at least one of a label of a reference image, text information in the reference image, and attribute information of the reference image, and determines at least one of the label of the reference image, text information in the reference image, and attribute information of the reference image as a second search keyword.

[0143] Step 1305: The terminal determines a third image matching the second search keyword from the image set based on the second search keyword and a second search graph of images in the image set.

[0144] The process by which the terminal determines a third image matching the second search keyword from the image set based on the second search keyword and the second search graph of images in the image set is similar to the process by which the terminal determines a first image matching the first search keyword from the image set based on the first search keyword and the second search graph of images in the image set, and will not be repeated here.

[0145] In the embodiment of the present application, not only can image search be performed by searching keywords, but also the ability to search by image is added, so that users can find some similar images based on reference images, thereby enriching the image search function. In addition, by searching by image, users are given a more convenient search experience.

[0146] Please refer to Fig.14 , which shows a flowchart of an image search method shown in an exemplary embodiment of the present application. Fig.14 , the method comprising:

[0147] Step 1401: For any image in the image set, the terminal determines a label of the image.

[0148] In some embodiments, this step is the same as step 201 and will not be repeated here.

[0149] Step 1402: Based on the label of the image, the terminal determines a first search graph corresponding to the label from the full search graph, the first search graph includes the label and multiple extended information corresponding to the label, and the full search graph includes the first search graph corresponding to multiple preset labels.

[0150] In some embodiments, this step is the same as step 202 and will not be repeated here.

[0151] Step 1403: Based on the image, the terminal adaptively configures the first search graph to obtain a second search graph.

[0152] In some embodiments, this step is the same as step 203 and will not be repeated here.

[0153] Step 1404: When the search information is received, based on the search information and a second search graph of images in the image set, determine a first image matching the search information from the image set.

[0154] In some embodiments, this step is the same as step 204; alternatively, this step can be implemented through steps 704-706; alternatively, this step can be implemented through steps 1006-1007, which will not be described in detail here.

[0155] Step 1405: When the first image searched out does not satisfy the condition, the terminal determines at least one associative word of the search information.

[0156] Sometimes the result of an image search may not meet the requirements, and another search can be performed; accordingly, after the terminal displays the first image, in response to the image search button being triggered, the terminal determines that the first image searched out does not meet the conditions. At this time, the terminal determines at least one associative word of the search information (for example, the first search keyword) and displays at least one associative word for the user to select. The user can select an associative word from the at least one associative word and trigger it to the terminal, and then execute step 1406.

[0157] For example, continue to refer to Fig. 9 In search interface 3, after the user searches for "dog", related words can be recommended to the user. After searching for images of "dog", there may be too many or no desired images. You can further search for associated words related to "dog" in the image collection, such as "people" and "grass". The user can select one or more for supplementary search, so as to quickly find the image he wants. In the image search results, you can further search for related tags, such as search interface 6.

[0158] Step 1406: The terminal performs a supplementary search based on at least one associative word to obtain a second image.

[0159] In some embodiments, the terminal determines the selected associative word among at least one associative word, and obtains the second image associated with the associative word. In other embodiments, the terminal determines the selected associative word among at least one associative word, performs a supplementary search from multiple first images obtained in the previous search, and obtains the second image.

[0160] In the embodiment of the present application, the user can click on the associated word based on the previous search to perform a more precise search, thereby obtaining a faster and better image search experience. Furthermore, by searching again, the user can further obtain more accurate images within the result range.

[0161] Please refer to Fig.15 , which shows a block diagram of an image search device shown in an exemplary embodiment of the present application, the device includes:

[0162] A first determining module 1501 is used to determine a label of an image for any image in the image set;

[0163] A second determination module 1502 is used to determine a first search graph corresponding to the label from the full search graph based on the label of the image, wherein the first search graph includes the label and a plurality of extended information corresponding to the label, and the full search graph includes a first search graph corresponding to a plurality of preset labels;

[0164] A configuration module 1503 is used to adaptively configure the first search graph based on the image to obtain a second search graph;

[0165] The search module 1504 is configured to determine, when receiving the search information, a first image matching the search information from the image set based on the search information and a second search graph of images in the image set.

[0166] In some embodiments, configuration module 1503 is used to, for any extended information in the first search graph, determine the extended information as a search object in the first search graph when the image is associated with the extended information; and determine the extended information as a non-search object in the first search graph when the image is not associated with the extended information, thereby obtaining a second search graph.

[0167] In some embodiments, the apparatus further comprises:

[0168] a third determination module, for determining that the image is associated with the extended information when the extended information is a synonym of the label, or when the extended information is a hypernym of the label, or when the extended information is a hyponym of the label and the image includes image content corresponding to the hyponym, or when the extended information is a correction word of the label, or when the extended information is a complement word of the label, or when the extended information is an associative word of the label and the image includes image content corresponding to the associative word, or when the extended information is a recommended word of the label and the image includes image content corresponding to the recommended word, or when the extended information is a custom word of the label;

[0169] The fourth determination module is used to determine that the image is not associated with the extended information when the extended information is a hyponym of the tag but the image does not include the image content corresponding to the hyponym, or when the extended information is a recommended word of the tag but the image does not include the image content corresponding to the recommended word, or when the extended information is an associative word of the tag but the image does not include the image content corresponding to the associative word.

[0170] In some embodiments, the apparatus further comprises:

[0171] An expansion module, for expanding any preset tag among the multiple preset tags based on multiple expansion words to obtain multiple expansion information corresponding to the preset tag;

[0172] A generating module, configured to generate a first search graph corresponding to the preset tag based on the preset tag and a plurality of extended information corresponding to the preset tag;

[0173] The composition module is used to combine the first search graphs corresponding to multiple preset tags into a full search graph.

[0174] In some embodiments, the plurality of expanded words include a plurality of synonyms, hypernyms, hyponyms, associative words, error correction words, completion words, recommended words, and custom words;

[0175] The extension module is used to determine multiple synonyms, hypernyms, hyponyms, associative words, completion words, recommended words and custom words of the preset tag; and to form multiple synonyms, hypernyms, hyponyms, associative words, completion words, recommended words and custom words of the preset tag into multiple extension information.

[0176] In some embodiments, the search information is a first search keyword; the search module 1504 is used to determine a recommended index based on the first search keyword and a second search graph of images in the image set; display the recommended index; in response to the recommended index being triggered, obtain a first image corresponding to the recommended index.

[0177] In some embodiments, the search module 1504 is used to determine the index configuration information of the currently logged-in account, where the index configuration information is used to represent the configuration information for determining the recommended index when performing an image search; based on the search configuration information and the first search keyword, the recommended index is determined from the second search graph of the images in the image collection.

[0178] In some embodiments, the apparatus further comprises:

[0179] A display module, used to display a setting interface when the image search function is started for the first time, and the setting interface is used to set index configuration information;

[0180] The acquisition module is used to obtain the index configuration information based on the settings interface and store the index configuration information. The index configuration information is the default index configuration information or the index configuration information customized by the current login account;

[0181] The search module 1504 is used to obtain stored index configuration information.

[0182] In some embodiments, the index configuration information includes at least one of a label probability threshold, a priority level of multiple extended information, and a first identifier, the first identifier being used to indicate whether to use text information in the image as a search object;

[0183] A search module 1504 is configured to, for any label of an image in the image set, delete the label if the probability of the label is less than a label probability threshold, obtain a third search graph of any image in the image set, and determine a recommended index from the third search graph of the image in the image set based on the first search keyword;

[0184] A search module 1504 is configured to perform a matching operation on the tags and the multiple extended information in sequence with the first search keyword based on the priority levels of the tags and the priority levels of the multiple extended information included in the second search graph of the images in the image set, so as to obtain a recommendation index;

[0185] The search module 1504 is used to determine a recommended index from the text information of the image in the image set and the second search graph based on the first search keyword when the first identifier is used to indicate that the text information in the image is used as the search object.

[0186] In some embodiments, the search module 1504 is used to determine the attribute information of the images in the image set; based on the first search keyword, determine the recommended index from the text information, attribute information and the second search map of the images in the image set.

[0187] In some embodiments, the search module 1504 is used to determine at least one associative word of the search information when the first image searched out does not meet the condition; and perform a supplementary search based on the at least one associative word to obtain a second image.

[0188] In some embodiments, the search information is a reference image;

[0189] The search module 1504 is used to determine a second search keyword based on the reference image; and determine a third image matching the second search keyword from the image set based on the second search keyword and a second search graph of images in the image set.

[0190] In the embodiment of the present application, on the one hand, the first search graph includes the label of the image and multiple extended information corresponding to the label, and the multiple extended information can enrich the information content of the image search. On the other hand, the second search graph is obtained by adaptively configuring the first search graph based on the image; therefore, the present application can establish a personalized second search graph. Based on the above discussion, it can be seen that performing image search based on the second search graph can improve the accuracy of image search.

[0191] It should be noted that the image search device provided in the above embodiment only uses the division of the above functional modules as an example when performing image search. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the user terminal is divided into different functional modules to complete all or part of the functions described above. In addition, the image search device provided in the above embodiment and the image search method embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0192] Please refer to Fig.16 , which shows a block diagram of a terminal 1600 according to an exemplary embodiment of the present application. The terminal 1600 in the present application may include one or more of the following components: a processor 1610 , a memory 1620 , and a display screen 1630 .

[0193] The processor 1610 may include one or more processing cores. The processor 1610 uses various interfaces and lines to connect the various parts of the entire terminal 1600, and executes various functions and processes data of the terminal 1600 by running or executing instructions, programs, code sets or instruction sets stored in the memory 1620, and calling data stored in the memory 1620. Optionally, the processor 1610 can be implemented in at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), and programmable logic array (PLA). The processor 1610 can integrate one or more combinations of a central processing unit (CPU), a graphics processing unit (GPU), a neural network processor (NPU), and a modem. Among them, the CPU mainly processes the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing the content that needs to be displayed on the display screen 1630; the NPU is used to implement artificial intelligence (AI) functions; and the modem is used to process wireless communications. It is understandable that the above-mentioned modem may not be integrated into the processor 1610, but may be implemented by a separate chip.

[0194] The memory 1620 may include a random access memory (RAM) or a read-only memory (ROM). Optionally, the memory 1620 includes a non-transitory computer-readable storage medium. The memory 1620 may be used to store instructions, programs, codes, code sets, or instruction sets. The memory 1620 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the following various method embodiments, etc.; the data storage area may store data (such as audio data, a phone book), etc., created according to the use of the terminal 1600.

[0195] The display screen 1630 is a display component for displaying a user interface. Optionally, the display screen 1630 is a display screen with a touch function, through which the user can use any suitable object such as a finger, a touch pen, etc. to perform touch operations on the display screen 1630.

[0196] The display screen 1630 is usually arranged on the front panel of the terminal 1600. The display screen 1630 can be designed as a full screen, a curved screen, a special-shaped screen, a double-sided screen or a folding screen. The display screen 1630 can also be designed as a combination of a full screen and a curved screen, a combination of a special-shaped screen and a curved screen, etc., which is not limited in this embodiment.

[0197] In addition, those skilled in the art can understand that the structure of the terminal 1600 shown in the above drawings does not constitute a limitation on the terminal 1600, and the terminal 1600 may include more or fewer components than shown, or combine certain components, or arrange the components differently. For example, the terminal 1600 also includes an audio acquisition device, a speaker, a radio frequency circuit, an input unit, a sensor, an audio circuit, a wireless fidelity (Wireless Fidelity, Wi-Fi) module, a power supply, a Bluetooth module and other components, which will not be repeated here.

[0198] An embodiment of the present application further provides a computer-readable medium, which stores at least one program code, and the at least one program code is loaded and executed by the processor to implement the image search method shown in the above embodiments.

[0199] An embodiment of the present application further provides a computer program product, which stores at least one program code, and the at least one program code is loaded and executed by the processor to implement the image search method shown in the above embodiments.

[0200] In some embodiments, the computer program product involved in the embodiments of the present application may be deployed and executed on a user terminal, or on multiple user terminals located at one location, or on multiple user terminals distributed at multiple locations and interconnected by a communication network. Multiple user terminals distributed at multiple locations and interconnected by a communication network may constitute a blockchain system.

[0201] Those skilled in the art should be aware that in one or more of the above examples, the functions described in the embodiments of the present application can be implemented with hardware, software, firmware, or any combination thereof. When implemented using software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium. Computer-readable media include computer storage media and communication media, wherein the communication media include any media that facilitates the transmission of a computer program from one place to another. The storage medium can be any available medium that a general or special-purpose computer can access.

[0202] The above description is only an optional embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. An image search method, characterized in that: The method comprises: For any image in the image set, determining a label of the image; Based on the label of the image, determining a first search graph corresponding to the label from the full search graph, wherein the first search graph includes the label and a plurality of extended information corresponding to the label, and the full search graph includes a first search graph corresponding to a plurality of preset labels; Based on the image, adaptively configuring the first search graph to obtain a second search graph; When the search information is received, a first image matching the search information is determined from the image set based on the search information and a second search graph of images in the image set.

2. The method according to claim 1, characterized in that The adaptively configuring the first search graph based on the image to obtain a second search graph includes: For any extended information in the first search graph, when the image is associated with the extended information, the extended information is determined as a search object in the first search graph; when the image is not associated with the extended information, the extended information is determined as a non-search object in the first search graph to obtain the second search graph.

3. The method according to claim 2, characterized in that The method further comprises: In a case where the extended information is a synonym of the label, or in a case where the extended information is a hypernym of the label, or in a case where the extended information is a hyponym of the label and the image includes image content corresponding to the hyponym, or in a case where the extended information is a correction word of the label, or in a case where the extended information is a complement word of the label, or in a case where the extended information is an associative word of the label and the image includes image content corresponding to the associative word, or in a case where the extended information is a recommended word of the label and the image includes image content corresponding to the recommended word, or in a case where the extended information is a custom word of the label, it is determined that the image is associated with the extended information; When the extended information is a hyponym of the label but the image does not include image content corresponding to the hyponym, or when the extended information is a recommended word of the label but the image does not include image content corresponding to the recommended word, or when the extended information is an associative word of the label but the image does not include image content corresponding to the associative word, it is determined that the image is not associated with the extended information.

4. The method according to claim 1, characterized in that: The method further comprises: For any preset tag among the multiple preset tags, based on multiple extension words, the preset tag is expanded to obtain multiple extension information corresponding to the preset tag; Based on the preset tag and the multiple extended information corresponding to the preset tag, generating a first search graph corresponding to the preset tag; The first search graphs corresponding to the multiple preset tags are combined into the full search graph.

5. The method according to claim 4, characterized in that The multiple expanded words include multiple words selected from the group consisting of synonyms, hypernyms, hyponyms, associative words, error correction words, completion words, recommended words, and custom words; The step of expanding the preset tag based on the multiple extended words to obtain multiple extended information corresponding to the preset tag includes: Determine a plurality of synonyms, hypernyms, hyponyms, associative words, completion words, recommended words, and custom words of the preset tag; The plurality of extended information are composed of a plurality of synonyms, hypernyms, hyponyms, associative words, completion words, recommended words and user-defined words of the preset tags.

6. The method according to claim 1, characterized in that The search information is a first search keyword; and the second search graph based on the search information and the images in the image set, determining a first image matching the search information from the image set, comprises: Determining a recommendation index based on the first search keyword and a second search graph of images in the image set; Displaying the recommended index; In response to the recommendation index being triggered, a first image corresponding to the recommendation index is acquired.

7. The method according to claim 6, characterized in that The determining of a recommendation index based on the first search keyword and a second search graph of images in the image set includes: Determine index configuration information of the current login account, where the index configuration information is used to indicate configuration information for determining a recommended index when performing an image search; Based on the search configuration information and the first search keyword, a recommendation index is determined from a second search graph of images in the image collection.

8. The method according to claim 7, characterized in that The method further comprises: When the image search function is started for the first time, a setting interface is displayed, wherein the setting interface is used to set the index configuration information; Acquire the index configuration information set based on the setting interface, and store the index configuration information, where the index configuration information is the default index configuration information or the index configuration information customized by the current login account; The step of determining the index configuration information of the current login account includes: The stored index configuration information is obtained.

9. The method according to any one of claims 6 to 8, characterized in that: The index configuration information includes at least one of a label probability threshold, priority levels of multiple extended information, and a first identifier, wherein the first identifier is used to indicate whether to use text information in the image as a search object; The determining, based on the search configuration information and the first search keyword, a recommended index from a second search graph of images in the image set includes at least one of the following implementations: For any label of an image in the image set, if the probability of the label is less than the label probability threshold, delete the label to obtain a third search graph of any image in the image set, and determine a recommended index from the third search graph of images in the image set based on the first search keyword; Based on the priority levels of tags and the priority levels of multiple extended information included in the second search graph of the images in the image set, sequentially performing matching operations on the tags and the multiple extended information with the first search keyword to obtain a recommendation index; In the case where the first identifier is used to indicate the use of text information in an image as a search object, the recommended index is determined from the text information of the images in the image set and the second search graph based on the first search keyword.

10. The method according to claim 9, characterized in that The determining the recommended index from text information of the images in the image set and a second search graph based on the first search keyword includes: Determining attribute information of images in the image set; Based on the first search keyword, the recommendation index is determined from text information, attribute information and a second search graph of the images in the image set.

11. The method according to claim 1 or 6, characterized in that: The method further comprises: In the case that the first image searched out does not satisfy the condition, determining at least one associative word of the search information; A supplementary search is performed based on the at least one associative word to obtain a second image.

12. The method according to claim 1, characterized in that The search information is a reference image; The determining, based on the search information and a second search graph of images in the image set, a first image matching the search information from the image set comprises: Based on the reference image, determining a second search keyword; Based on the second search keyword and a second search graph of images in the image set, a third image matching the second search keyword is determined from the image set.

13. An image search device, characterized in that: The device comprises: A first determination module, configured to determine a label of any image in the image set; A second determination module is used to determine, based on the label of the image, a first search map corresponding to the label from the full search map, wherein the first search map includes the label and a plurality of extended information corresponding to the label, and the full search map includes a first search map corresponding to a plurality of preset labels; A configuration module, configured to adaptively configure the first search graph based on the image to obtain a second search graph; The search module is used to determine a first image matching the search information from the image set based on the search information and a second search graph of images in the image set when the search information is received.

14. A terminal, characterized in that: The terminal includes one or more processors and one or more memories, wherein at least one program code is stored in the one or more memories, and the at least one program code is loaded and executed by the one or more processors to implement the image search method according to any one of claims 1 to 12.

15. A computer-readable storage medium, characterized in that: The storage medium stores at least one program code, and the at least one program code is loaded and executed by the processor to implement the image search method according to any one of claims 1 to 12.

16. A computer program product, characterized in that The computer program product stores at least one program code, and the at least one program code is used to be executed by a processor to implement the image search method according to any one of claims 1 to 12.

Images