Image searching method and device

By segmenting and generating matching images, the problems of inaccuracy and efficiency of image search in the prior art are solved, and the pictures that are needed by users are efficiently searched in electronic devices.

CN119046488APending Publication Date: 2024-11-29VIVO MOBILE COMM CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411007338.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-25
Publication Date
2024-11-29

AI Technical Summary

Technical Problem

In the prior art, when a user searches for images, due to limited image information, the input search terms may not be able to accurately match the corresponding pictures, resulting in low accuracy and efficiency in searching pictures, and users need to repeatedly enter search terms. .

Method used

By segmenting the search terms input by the user, at least two word participle results are generated, and an image matching the search terms is generated based on these word participle results, thereby finding a matching target image in the image set.

Benefits of technology

It improves the accuracy and efficiency of electronic devices when searching pictures, avoids the repeated search process caused by the inability to match the search terms, and directly finds target images similar or the same as user input.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119046488A_ABST
    Figure CN119046488A_ABST
Patent Text Reader

Abstract

The invention discloses an image searching method and device, and belongs to the technical field of electronics. The method comprises the following steps: segmenting a search word input by a user to obtain at least two word segmentation results; based on the at least two word segmentation results, generating a first image matched with the search word; and based on the first image, determining a target image matched with the first image from a first image set.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application belongs to the field of electronic technology, and specifically relates to an image search method and a device thereof. Background Art

[0002] From daily life records to work data charts to artistic photos, a large number of pictures have become an indispensable part of people's lives. Users often need to search for pictures in order to quickly and accurately find the pictures they need from these massive pictures in electronic devices.

[0003] When a user currently searches for an image, he or she needs to enter a search term in a picture management application (such as an album) in the electronic device so that the electronic device can identify the information of the image, such as shooting time, shooting location, image tags, etc., through the picture management application, and then compare the recognition result with the search term entered by the user to determine the required image.

[0004] However, due to the limited information of the picture, the search term entered by the user may not match the corresponding picture information, so the electronic device cannot accurately find the required picture. The user then needs to enter the search term again to search for the picture, resulting in low accuracy and efficiency of the electronic device in finding pictures. Summary of the invention

[0005] The purpose of the embodiments of the present application is to provide an image search method, device, electronic device and storage medium, which can improve the accuracy and efficiency of electronic devices in searching for pictures required by users.

[0006] In a first aspect, an embodiment of the present application provides an image search method, the method comprising: segmenting a search term input by a user to obtain at least two segmentation results; based on the at least two segmentation results, generating a first image matching the search term; based on the first image, determining a target image matching the first image from a first image set.

[0007] In a second aspect, an embodiment of the present application provides an image search device, which includes: a segmentation module, a generation module, and a determination module. The segmentation module is used to segment a search term input by a user to obtain at least two segmentation results. The generation module is used to generate a first image matching the search term based on the at least two segmentation results obtained by the segmentation module. The determination module is used to determine a target image matching the first image from a first image set based on the first image generated by the generation module.

[0008] In a third aspect, an embodiment of the present application provides an electronic device, which includes a processor and a memory, wherein the memory stores programs or instructions that can be run on the processor, and when the program or instructions are executed by the processor, the steps of the method described in the first aspect are implemented.

[0009] In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by a processor, the steps of the method described in the first aspect are implemented.

[0010] In a fifth aspect, an embodiment of the present application provides a chip, comprising a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run a program or instruction to implement the method described in the first aspect.

[0011] In a sixth aspect, an embodiment of the present application provides a computer program product, which is stored in a storage medium and is executed by at least one processor to implement the method described in the first aspect.

[0012] In an embodiment of the present application, the search term input by the user can be segmented to obtain at least two segmentation results, and based on the at least two segmentation results, a first image matching the search term can be generated, and then based on the first image, a target image matching the first image can be determined from the first image set. In this solution, since the electronic device can segment the search term to obtain at least two segmentation results that are more conducive to image generation, that is, keywords that can generate images are obtained, and then corresponding images are generated according to these keywords, that is, the first image matching the search term, the electronic device can compare the first image with the images in the first image set to find the target image matching the first image, that is, the target image found is the same or very similar to the image generated by the search term input by the user, avoiding the inability to match the corresponding image directly based on the search term input by the user, and the user does not need to search again, so that the electronic device can accurately find the required image, improving the accuracy and efficiency of the electronic device in finding images. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 It is one of the flow charts of the image search method provided in the embodiment of the present application;

[0014] Figure 2 This is the second flowchart of the image search method provided in the embodiment of the present application;

[0015] Figure 3 This is the third flowchart of the image search method provided in the embodiment of the present application;

[0016] Figure 4This is the fourth flowchart of the image search method provided in the embodiment of the present application;

[0017] Figure 5 is a schematic diagram of an image search device provided in an embodiment of the present application;

[0018] Figure 6 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0019] Figure 7 It is a schematic diagram of the hardware structure of the electronic device provided in the embodiment of the present application. DETAILED DESCRIPTION

[0020] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0021] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0022] The terms "at least one (item)", "at least one of" and the like in this application refer to any one, any two or a combination of more than two of the objects included therein. For example, at least one (item) of a, b, and c can be represented by: "a", "b", "c", "a and b", "a and c", "b and c" and "a, b and c", where a, b, and c can be single or multiple. Similarly, "at least two (items)" refers to two or more, and its meaning is similar to that of "at least one (item)".

[0023] The image search method provided in the embodiment of the present application is described in detail below through specific embodiments and their application scenarios in conjunction with the accompanying drawings.

[0024] The embodiment of the present application can be applied to a scenario where an electronic device searches for pictures. Specifically, the electronic device searches for images matching the search words input by the user.

[0025] Usually, when searching for pictures, users need to enter search terms in a picture management application (such as an album) in an electronic device, so that the electronic device can identify the information of the picture, such as the shooting time, shooting location, picture tags, etc., through the picture management application, and then compare the recognition result with the search term entered by the user to determine the required picture. However, due to the limited information of the picture, the search term entered by the user may not match the corresponding picture information, so the electronic device cannot accurately find the required picture, and the user needs to enter the search term again to search for the picture, resulting in low accuracy and efficiency of the electronic device in finding pictures.

[0026] The embodiment of the present application provides an image search method, which can segment the search word input by the user to obtain at least two segmentation results, and based on the at least two segmentation results, generate a first image matching the search word, and then determine the target image matching the first image from the first image set based on the first image. In this solution, since the electronic device can segment the search word to obtain at least two segmentation results that are more conducive to image generation, that is, obtain keywords that can generate images, and then generate corresponding images based on these keywords, that is, the first image matching the search word, the electronic device can compare the first image with the images in the first image set to find the target image matching the first image, that is, the target image found is the same or very similar to the image generated by the search word input by the user, avoiding the inability to match the corresponding image directly based on the search word input by the user, and the user does not need to search again, so that the electronic device can accurately find the required image, improving the accuracy and efficiency of the electronic device in finding images.

[0027] The execution subject of the image search method provided in the embodiment of the present application may be an image search device, or an electronic device, or a functional module in an electronic device. The technical solution provided in the embodiment of the present application is described below using an electronic device as an example.

[0028] Figure 1 A flowchart of an image search method provided by an embodiment of the present application is shown. Figure 1 As shown, the image search method provided in the embodiment of the present application may include the following steps 201 to 203.

[0029] Step 201: The electronic device segments the search term input by the user to obtain at least two segmentation results.

[0030] In the embodiment of the present application, the above-mentioned search words are search keywords input by the user when searching for images, so that the electronic device can generate images matching the search words.

[0031] In the embodiment of the present application, the search term S is composed of x single characters c1, c2, c3...c x S can be divided into m meaningful words W1, W2, W3…W m , so that S = W1W2…W m .

[0032] Optionally, in an embodiment of the present application, the search terms may include but are not limited to at least one of the following characters: letters, numbers, punctuation marks, special symbols, spaces, Chinese characters, characters unique to other writing systems, etc. In actual use, since there are spaces between words in English or French, etc., each word can be conveniently divided into individual characters, while there are no spaces between Chinese characters and other characters to separate characters, it is necessary to insert spaces between each Chinese character to treat each Chinese character as a separate character, so as to facilitate the electronic device to combine characters into character pairs. For example, "a red rose blooming in the sun" can be obtained by inserting spaces to obtain "a red rose blooming in the sun" to treat each Chinese character as a separate character.

[0033] In an embodiment of the present application, the above-mentioned search term may include at least one character, and the electronic device may segment the search term input by the user, segmenting at least one character in the search term into at least two words with practical meanings, and obtain at least two word segmentation results, so that the electronic device can generate a matching image based on the at least two word segmentation results.

[0034] In the embodiment of the present application, the at least two word segmentation results are word segmentation results obtained by the electronic device segmenting the search word so as to maximize the joint probability of the sequence of the obtained words, which can be expressed by the following formula (1):

[0035]

[0036] Among them, P(W m ) is the joint probability of the mth word.

[0037] For example, suppose that all words in 30 issues of newspapers are counted, there are Q words in total, and the word rose obtained by combining the two characters "rose" appears P times, then the joint probability of rose is equal to P / Q, and the frequency of rose appearance is P. The Q words here can be understood as the preset word set described in the embodiment of the present application.

[0038] It should be noted that maximizing the joint probability means maximizing the frequency of the words obtained by character combination in the preset word set, so as to accurately segment the search words and obtain at least two word segmentation results.

[0039] In the embodiments of the present application, the above-mentioned preset word set, also known as a corpus, is a database containing a large amount of text. These texts are usually organized and have a predefined format and markup, such as a Chinese corpus, a Chinese word-segmented corpus, etc. The main function of the corpus is to serve as a basic resource for natural language processing systems, helping researchers and developers understand and simulate the usage of human language. Thus, in the embodiments of the present application, the corpus is used as a reference object for the occurrence frequency of character pairs.

[0040] Optionally, in the embodiments of the present application, in combination with Figure 1 , as Figure 2 shown, the above-mentioned step 201 can be specifically implemented by the following steps 301 to 304.

[0041] Step 301: The electronic device performs N combinations on the characters in the search term.

[0042] Where N is an integer greater than 1.

[0043] In the embodiments of the present application, the electronic device can use a word-segmentation model to insert spaces between each character in the search term to split the search term into multiple individual characters, so as to perform combinations on all characters.

[0044] Exemplarily, the search term input by the user is "a red rose blooming in the sun". The electronic device can insert spaces between each Chinese character in the search term to get "a red rose blooming in the sun", thereby splitting the search term into characters such as "一", "朵", "在", "阳", "光", "下", "绽", "放", "的", "红", "玫", "瑰", and perform N combinations on these characters.

[0045] Step 302: The electronic device obtains at least two word-segmentation results based on the character pairs obtained from the Nth combination.

[0046] Optionally, in the embodiments of the present application, the electronic device can use the character pairs obtained from the Nth combination as at least two word-segmentation results. Or, the electronic device can select the character pairs corresponding to content words or content meanings from the character pairs obtained from the Nth combination as at least two word-segmentation results.

[0047] Step 303: For the first combination, the electronic device combines every two adjacent characters among all the characters to obtain at least one character pair.

[0048] In the embodiments of the present application, the above-mentioned character pair is the result of combining two characters. For example, the characters "一" and "朵" can be combined into a character pair ("一朵").

[0049] In an embodiment of the present application, during the first combination, the electronic device can arrange the single characters obtained by segmenting the search term in the order of input, so that every two adjacent characters are combined into a character pair, so that the electronic device can determine the frequency of occurrence of the character pair.

[0050] Example 1, the search term input by the user is "a red rose blooming in the sun", the electronic device can split the search term into "one", "flower", "in", "sun", "light", "under", "bloom", "put", "of", "red", "rose", "rose", and then the electronic device can combine all the characters for the first time, that is, combine each adjacent character into a character pair, and obtain the following results: ("one"), ("flower in"), ("in the sun"), ("sunshine"), ("under the light"), ("bloom under"), ("bloom"), ("put"), ("of red"), ("red rose"), ("rose").

[0051] Step 304: For the i-th combination, the electronic device determines the first character pair that appears most frequently in the preset word set from the character pairs obtained from the i-1-th combination, and combines every two adjacent characters of the remaining characters in all characters, as well as the first character pair and the characters adjacent to the first character pair.

[0052] Here, i=2, ..., N. It should be noted that, for each combination (starting from the second time), the remaining characters refer to all characters except the characters included in the first character pair determined each time.

[0053] It can be understood that starting from the second combination, each time a combination is performed, the electronic device can first determine the first character pair with the highest frequency of occurrence in the preset word set from the character pairs obtained in the previous combination, regard the first character pair as one character, and then combine every two adjacent characters of all current characters.

[0054] In an embodiment of the present application, the electronic device can obtain all characters in the search word, and combine every two adjacent characters of all the characters to obtain X character pairs, and then the electronic device can determine the first character pair that appears most frequently in the preset word set from the X character pairs, and combine every two adjacent characters of the remaining characters of all the characters, and combine the first character pair with the characters adjacent to the first character pair, to obtain Y character pairs, so that the electronic device can determine the second character pair that appears most frequently in the preset word set from the Y character pairs, and combine every two adjacent characters of the remaining characters of all the characters, and combine the second character pair with the characters adjacent to the second character pair, to obtain Z character pairs, and so on, until the number of combinations reaches the preset number, and finally obtain at least two word segmentation results based on the character pairs obtained after the combination.

[0055] Combined with Example 1, as shown in Example 2, after the first combination, after the electronic device combines each adjacent character among all characters to obtain at least one character pair, the electronic device can count the frequency of occurrence of the at least one character pair in the preset word set. For example, Count("一朵") = 15, indicating that "一朵" appears 15 times in the corpus, Count("朵在") = 8, Count("在阳") = 7, Count("阳光") = 20, Count("光下") = 4, Count("下绽") = 2, Count("绽放") = 18, Count("放的") = 4, Count("的红") = 4, Count("红玫") = 10, Count("玫瑰") = 25.

[0056] At the second combination, the electronic device can determine the first character pair with the highest frequency of occurrence in the preset word set from the character pairs obtained in the first combination. That is, the frequency of occurrence of ("玫瑰") is the largest, and the first character pair is "玫瑰", which can be regarded as a new character. At this time, all the current characters are updated to "一", "朵", "在", "阳", "光", "下", "绽", "放", "的", "红", "玫瑰", and then the electronic device can combine every two adjacent characters among the current all characters into a character pair, obtaining ("一朵"), ("朵在"), ("在阳"), ("阳光"), ("光下"), ("下绽"), ("绽放"), ("放的"), ("的红"), ("红玫瑰").

[0057] At the third combination, the electronic device can count the frequency of occurrence of ("红玫瑰") obtained in the second combination in the preset word set, getting Count("红玫瑰") = 17, and the frequencies of occurrence of the remaining characters remain unchanged. Then, the frequency of occurrence of ("阳光") is the largest, and the first character pair is "阳光", which can be regarded as a new character. At this time, all the current characters are updated to "一", "朵", "在", "阳光", "下", "绽", "放", "的", "红", "玫瑰", and then the electronic device can combine every two adjacent characters among the current all characters into a character pair, obtaining ("一朵"), ("朵在"), ("在阳光"), ("阳光下"), ("下绽"), ("绽放"), ("放的"), ("的红"), ("红玫瑰"). At the fourth combination, the electronic device can count the frequencies of occurrence of ("在阳光") and ("阳光下") obtained in the third combination in the preset word set, determine the character pair with the highest frequency of occurrence in the preset word set, and continue to combine, and so on until the Nth combination is reached.

[0058] Combined with Example 2, as shown in Example 3, assume that N is 6. In the sixth combination, the electronic device can count the frequencies of all character pairs obtained in the fifth combination in the preset word set, wherein the characters obtained in the fifth combination are: "a", "in", "sunshine", "under", "bloom", "of", "red roses", and the character pairs composed of all adjacent characters and their corresponding frequencies are: Count ("a") = 8, Count ("in the sun") = 7, Count ("under the sun") = 10, Count ("bloom under") = 2, Count ("put") = 4, Count ("red roses") = 4, wherein ("under the sun") has the highest frequency of occurrence, and the first character pair is "under the sun", which can be regarded as a new character. At this time, the number of combinations reaches the preset number of 6 times, and the electronic device can determine the characters obtained after the sixth combination, that is, "a", "in", "sunshine", "bloom", "of", "red roses" as the above at least two word segmentation results, or can determine "a", "sunshine", "bloom", "red roses" as the above at least two word segmentation results.

[0059] In this way, the electronic device can divide the search word into single characters, and then combine them into multiple character pairs. By using the frequency of appearance of these character pairs in a preset character set, the most likely word segmentation results can be screened out, thereby dividing the search word into multiple words with practical meanings, so that the electronic device can generate an image based on the words in the word segmentation results, thereby improving the matching degree between the generated image and the search word.

[0060] Step 202: The electronic device generates a first image matching the search term based on at least two word segmentation results.

[0061] It should be noted that the first image matching the search term can be understood as: the similarity (or correlation) between the search term and the description information of the first image is greater than or equal to a threshold, or the similarity (or correlation) between the search term and the image content of the first image is greater than or equal to a threshold.

[0062] In the embodiment of the present application, the electronic device can generate a first image matching the search term through a central image generation model based on at least two word segmentation results and a random noise parameter.

[0063] Optionally, in the embodiment of the present application, combined with Figure 1 ,like Figure 3 As shown, the above step 202 can be specifically implemented through the following steps 401 to 404.

[0064] Step 401: The electronic device converts at least two word segmentation results into numerical vectors through an image generation model.

[0065] In the embodiment of the present application, the above numerical vector represents the semantic information of the word segmentation result.

[0066] In an embodiment of the present application, the above-mentioned numerical vector can be referred to as a word segmentation vector result, or simply a word vector. After obtaining the word segmentation result, the electronic device can convert the word segmentation result into a word vector (which can be referred to as a word segmentation vector result) through a word vector model, and then input the word vector into a generation model in the image generation model to generate a first image. For example, if the word vector of "a flower" is represented by Emb (a flower), a 256-dimensional vector is used to represent it: Emb (a flower) = [12, 4, 434, 123, ...]. After converting each word segmentation result into a word vector, a word segmentation vector result w is obtained, so that the electronic device can generate a first image based on the word segmentation vector result w.

[0067] In the embodiment of the present application, the above-mentioned image generation model is also called the central image generation model, including a generation model (P), which can generate a central image according to the central point c, the word segmentation vector result w, the random noise parameter z, and the rectangular area R generated in the previous step. i-1 Generate the next rectangular area R i Its mathematical expression is: R i =P(z,R i-1 |c,w), where R i represents the next rectangular area, P represents the generation model, which needs to be trained, w represents the word segmentation vector result, z represents the random noise parameter, c represents the center point, R i-1 Represents the previous rectangular area.

[0068] Step 402: The electronic device generates a first image rectangular area based on the numerical vector and the random noise parameter.

[0069] Step 403: The electronic device extracts image features of the n-1th image rectangular region, and generates the nth image rectangular region based on the image features of the n-1th image rectangular region, the numerical vector and the random noise parameter.

[0070] Here, n is an integer greater than 1.

[0071] In the embodiment of the present application, the image feature includes at least one of the following: a color feature of the image, a texture feature of the image, a shape feature of the image, and a spatial relationship feature of the image.

[0072] Step 404: When the image formed by all the generated image rectangular regions matches the search term, the electronic device synthesizes all the image rectangular regions to obtain a first image.

[0073] In an embodiment of the present application, the electronic device can generate n rectangular image areas outward from a center point through a central image generation model based on at least two word segmentation results and random noise parameters. When generating a rectangular image area from the second to the nth time, it continues to generate outward based on the rectangular image area generated last time, and then determines whether the image formed by all the generated rectangular image areas matches the search term. If not, it continues to generate until the image formed by all the generated image rectangular areas matches the search term, then all the image rectangular areas are synthesized into a complete image to obtain a first image.

[0074] Exemplarily, the electronic device generates an image based on the word segmentation results "one", "in", "under the sun", "blooming", "of", and "red rose". When generating the rectangular image area for the first time, the electronic device can determine a center point of the image, and then based on the word segmentation result and random noise, through the central image generation model, start from the center point and generate a rectangular image area of ​​the rose outward, and the rectangular image area is a partial image of the first image. When generating the rectangular image area for the second time, the electronic device can, based on the word segmentation result and random noise, continue to generate a rectangular image area of ​​the branches and leaves of the rose outward on the basis of the rectangular image area of ​​the rose generated for the first time, and then merge the rectangular image area generated for the first time with the rectangular image area generated for the second time to obtain a rectangular image area with a rose with branches and leaves. At this time, if it is determined that the rectangular image area does not match the search term, the generation continues. When generating the rectangular image area for the third time, the electronic device can, based on the word segmentation results and random noise, continue to generate a rectangular image area with a sunlight effect outward on the basis of the rectangular image area of ​​the rose with branches and leaves generated for the second time, and then merge the rectangular image area generated for the second time with the rectangular image area generated for the third time to obtain a rectangular image area with a rose with branches and leaves in the sun. At this time, if it is determined that the rectangular image area matches the search term, the generation is stopped, and all the image rectangular areas are synthesized into a complete image to obtain the first image.

[0075] Optionally, in an embodiment of the present application, the electronic device may determine the resolution of the screen as the resolution of the first image.

[0076] Optionally, in an embodiment of the present application, the electronic device may determine the size of the first image according to the shooting size of the camera of the electronic device.

[0077] Optionally, in the embodiment of the present application, the electronic device may generate multiple image rectangular areas and then merge them into the first image, or may generate multiple image annular areas and merge them into the first image, or may be areas of other shapes. The embodiment of the present application is not specifically limited.

[0078] In this way, the electronic device can generate multiple image rectangular areas according to each word segmentation result, and after generating each image rectangular area, determine whether the image constituted by all the image rectangular areas matches the word segmentation result. If the constituted image does not match the word segmentation result, the image rectangular areas are regenerated until the image constituted by the generated image rectangular areas matches the word segmentation result, thereby improving the accuracy of the first image generation.

[0079] Optionally, the image search method provided in the embodiment of the present application further includes the following steps 501 and 502.

[0080] Step 501: For any image rectangular area among all the image rectangular areas, the electronic device obtains evaluation information based on at least two word segmentation results, any image rectangular area and any image rectangular area through a judgment model in an image generation model.

[0081] In the embodiment of the present application, the above evaluation information represents the authenticity of any rectangular area of ​​the image.

[0082] It should be noted that the embodiment of the present application is illustrated by taking an image rectangular region (ie any image rectangular region) as an example. After generating an image rectangular region each time, the electronic device may execute steps 501 and 502 to determine whether to regenerate the image rectangular region.

[0083] In the embodiment of the present application, the above-mentioned image generation model also includes a judgment model (C), which can evaluate the given center point c, the word segmentation vector w and the previous rectangular area R i-1 Under these conditions, the rectangular area R i The authenticity of is expressed mathematically as the following formula (2):

[0084] prob=C(R i |c,w,R i-1 ) Formula (2)

[0085] Among them, prob represents authenticity, C represents the judgment model, and R i represents a rectangular area, w represents the word segmentation vector result, c represents the center point, R i-1 Represents the previous rectangular area.

[0086] Exemplarily, taking the word segmentation results of "one", "in", "under the sun", "blooming", "of", and "red rose" as examples, the electronic device generates a first image based on the word segmentation results. After generating a rectangular image area of ​​a rose for the first time, the electronic device obtains evaluation information corresponding to the first image rectangular area through the judgment model in the image generation model based on the word segmentation result and the image rectangular area generated for the first time. After generating a rectangular image area of ​​the branches and leaves of a rose for the second time, the electronic device obtains evaluation information corresponding to the second image rectangular area through the judgment model in the image generation model based on the word segmentation result and the first image rectangular area and the second image area. And so on, each time a new rectangular image area is generated, the rectangular image area is evaluated to obtain evaluation information of the image rectangular area.

[0087] Step 502: The electronic device determines whether to regenerate any image rectangular area based on the evaluation information.

[0088] In an embodiment of the present application, when the authenticity represented by the nth evaluation information is greater than or equal to the preset degree, the next image rectangular area, i.e., the n+1th image rectangular area, is generated; when the authenticity represented by the nth evaluation information is less than the preset degree, the nth image rectangular area is regenerated, and the nth evaluation information is regenerated. If the authenticity represented by the nth evaluation information is still less than the preset degree, the nth image rectangular area is regenerated again, until the authenticity represented by the nth evaluation information is greater than or equal to the preset degree, and then the next image rectangular area is generated.

[0089] Exemplarily, taking the word segmentation results of "a", "in", "under the sun", "blooming", "of", and "red rose" as examples, the electronic device generates an image based on the word segmentation results. After generating a rectangular image area of ​​a rose for the first time, the electronic device obtains the first evaluation information corresponding to the first image rectangular area based on the word segmentation result and the image rectangular area generated for the first time through the judgment model in the image generation model. At this time, the electronic device judges the authenticity of the representation of the first evaluation information. If it is judged that the authenticity of the representation of the first evaluation information is less than a preset degree, the first image rectangular area is regenerated and the first evaluation information is regenerated, and then the authenticity of its representation is judged again, until it is judged that the authenticity of the representation of the first evaluation information is greater than or equal to the preset degree, then the second image rectangular area is generated. After generating a rectangular image area of ​​the branches and leaves of a rose for the second time, the electronic device obtains the second evaluation information corresponding to the second image rectangular area through the judgment model in the image generation model based on the word segmentation result and the first image rectangular area and the second image area. At this time, the electronic device judges the authenticity of the representation of the second evaluation information. If it is judged that the authenticity of the representation of the second evaluation information is less than a preset degree, the second image rectangular area is regenerated and the second evaluation information is regenerated, and then the authenticity of its representation is judged again until it is judged that the authenticity of the representation of the second evaluation information is greater than or equal to the preset degree, and then the third image rectangular area is generated. By analogy, each time a new rectangular image area is generated, the rectangular image area is evaluated to obtain evaluation information of the nth image rectangular area, and then the authenticity of the representation of the nth evaluation information is judged. When the authenticity of the representation of the nth evaluation information is less than a preset degree, the nth image rectangular area is regenerated and the nth evaluation information is regenerated, and then the authenticity of its representation is judged again, until the authenticity of the representation of the nth evaluation information is greater than or equal to the preset degree, and then the (n+1)th image rectangular area is generated.

[0090] In the embodiment of the present application, the electronic device can train the generation model (P) and the judgment model (C) based on the word segmentation vector result w through the training model to improve the matching degree between the generated image and the word segmentation result. The mathematical expression of the training model is as follows:

[0091]

[0092] Among them, T represents the word segmentation vector result.

[0093] In this way, when the electronic device generates an image rectangular area, it can determine the authenticity of the generated image rectangular area, thereby removing the image rectangular area with lower authenticity and generating an image rectangular area with higher authenticity, thereby improving the authenticity of the generated first image.

[0094] Step 203: The electronic device determines, based on the first image, a target image matching the first image from the first image set.

[0095] In an embodiment of the present application, the above-mentioned first image set may include but is not limited to any of the following situations: all images saved in the electronic device, all images in a folder of the electronic device, all images screened by the electronic device according to resolution and other categories, online images such as cloud albums, etc.

[0096] In the embodiment of the present application, the electronic device may compare the first image with each image in the first image set respectively, and determine a target image that matches the first image based on the degree of similarity.

[0097] It should be noted that the target image that matches the first image can be understood as: the similarity (or correlation degree) between the image information of the first image (including at least one of the image content and image parameters) and the image information of the target image is greater than or equal to a threshold (for example, 95%).

[0098] Optionally, in the embodiment of the present application, combined with Figure 1 ,like Figure 4 As shown, the above step 203 can be specifically implemented through the following steps 601 to 603.

[0099] Step 601: The electronic device calculates a first similarity value between the first image and each second image based on a feature vector of the first image, a feature vector of each second image in a first image set, and a preset number of feature values, and determines M images from the first image set whose first similarity values ​​are greater than or equal to a first threshold.

[0100] In the embodiment of the present application, the electronic device can quickly select M second images from the first image set to form a candidate image set based on the similarity value between the first image and each second image and the first threshold. This step does not pursue extremely high accuracy, but focuses on improving the recall rate to ensure that the final target image will not be missed in this step, and M is a positive integer.

[0101] In the embodiment of the present application, the electronic device calculates the first similarity value between the first image and each second image by the following formula (4):

[0102]

[0103] Among them, Similar represents the similarity between the generated image q and each image i in the album, a is the adjustment coefficient, which is generally 0.4, b is the adjustment coefficient, which is generally 0.6, f(q) represents the eigenvector value of image q, f(i) represents the eigenvector value of image i, which can be obtained by eigenvalue decomposition, k is the number of eigenvalues, · represents the vector dot product, and |·| represents the modulus of the vector.

[0104] In the embodiment of the present application, the principle by which the electronic device calculates the similarity between images based on the features of the images is that the image can be regarded as a three-dimensional matrix composed of three color channels, each color channel represents a matrix, and the elements of the matrix represent the pixel values ​​of the image. The principle of eigenvalue decomposition is: for an eigenvalue λ and the corresponding eigenvector v, they satisfy the equation Av=λv. Wherein A represents the matrix, and v represents the eigenvalue, which is f(q) in the above formula. The first image is compared with each second image by the above formula, and the top M second images with the highest similarity value to the first image are selected from the first image set.

[0105] Step 602: The electronic device calculates a second similarity value between each third image and each eigenvalue of the first image based on multiple eigenvalues ​​of the first image and multiple eigenvalues ​​of each third image in the M images, obtains a third similarity value between the first image and each third image, and determines L images from the M images whose third similarity values ​​are greater than or equal to the second threshold.

[0106] Optionally, in an embodiment of the present application, the above-mentioned multiple feature values ​​may include at least two of the following: color feature values, texture feature values, shape feature values, spatial relationships, etc.

[0107] In an embodiment of the present application, the above-mentioned multiple feature values ​​of the image represent multiple features of the image, such as color, texture, shape, etc. The electronic device can compare the multiple feature values ​​of the first image and each third image and compare the multiple features of the first image and each third image to obtain the similarity of each feature of the first image and each third image, and then determine the third similarity value of the first image and each third image based on the similarity of each feature and the weight of each feature, and determine L images from the M images whose third similarity values ​​are greater than or equal to the second threshold, where L is a positive integer less than or equal to M.

[0108] In the embodiment of the present application, the mathematical expression for weighted calculation based on the combination of color, texture, and shape is the following formula (5):

[0109] s(q,i)=w C *colorSim(C q ,C i )+w t *textureSim(T q,T i )+w s *shapeSim(S q ,S i ) Formula (5)

[0110] Among them, s(q,i) represents the similarity between image q and image i, w C is the weight of color similarity, colorSim is the color similarity, and the similarity of color histograms can be calculated using histogram intersection or Bhattacharyya distance, w t is the weight of texture similarity, textureSim represents texture similarity, and the similarity of texture features can be calculated using Euclidean distance or chi-square distance. s is the shape similarity weight, shapeSim is the shape similarity, and the similarity of shape features is calculated using Hausdorff distance or shape context. The above similarities can all be calculated using MATLAB.

[0111] Step 603: The electronic device calculates a fourth similarity value between each fourth image and the first image based on the deep features of each fourth image in the L images and the deep features of the first image, and determines an image from the L images whose fourth similarity value is greater than or equal to a third threshold to obtain a target image.

[0112] In an embodiment of the present application, the electronic device may use a deep learning model to extract deep features of an image, so that the electronic device can compare the first image with each fourth image through more refined deep features and complex similarity calculation methods to obtain a fourth similarity value between each fourth image and the first image, and then the electronic device may compare a third threshold with each fourth similarity value and determine the fourth image whose fourth similarity value is greater than or equal to the third threshold as the target image.

[0113] In the embodiment of the present application, the mathematical expression of the deep features of the image extracted by the deep feature model is as follows:

[0114]

[0115] Wherein, F(q) and F(i) represent the deep feature representation of the first image q and the fourth image i respectively, which can be extracted by the model. The deep features are represented by the parameters of the deep learning model, and a convolutional neural network is generally used to extract the features of the image.

[0116] In the embodiment of the present application, the number p of target images can be determined by the difference between each two similarities after sorting the similarities between each fourth image and the first image. Starting from the highest similarity value, when the difference between the similarity value corresponding to the fourth image a and the similarity value corresponding to the fourth image b is less than or equal to a preset threshold, the electronic device can determine the fourth image a and all fourth images with a similarity greater than that of the fourth image a as target images, and its mathematical expression is as shown in the following formula (7):

[0117]

[0118] Here, δ is a preset threshold, which is usually 0.2, and due to the limited size of the electronic device screen, p is at most 10.

[0119] Exemplarily, the similarity values ​​of the 7 fourth images to the first image are ordered as {0.85, 0.82, 0.81, 0.75, 0.53, 0.51, 0.49}, where 0.53 and 0.75>0.2. The electronic device can determine the fourth images corresponding to 0.75, 0.81, 0.82, and 0.85 as the target images, and p is determined to be 4.

[0120] In this way, the electronic device can compare the similarity values ​​of the first image with the images in the first image set three times through the feature vector of the image to the multiple feature values ​​of the image and then to the deep features of the image, so that the electronic device can determine the target image that matches the first image from the first image set, thereby improving the accuracy of the electronic device in determining the target image.

[0121] The embodiment of the present application provides an image search method. Since the electronic device can segment the search words to obtain at least two segmentation results that are more conducive to image generation, that is, obtain keywords that can generate images, and then generate corresponding images based on these keywords, that is, the first image that matches the search words, so that the electronic device can compare the first image with the images in the first image set to find a target image that matches the first image, that is, the target image found is an image that is identical to or extremely similar to the image generated by the search words input by the user, thereby avoiding the situation where the corresponding picture cannot be matched directly according to the search words input by the user, and the user does not need to search again. In this way, the electronic device can accurately find the required pictures, thereby improving the accuracy and efficiency of the electronic device in finding pictures.

[0122] The following is an explanation of the image search method provided in the embodiment of the present application through a specific implementation process.

[0123] Suppose the search term entered by the user is "a red rose blooming in the sun". The electronic device can obtain all the characters in the search term, namely "一", "朵", "在", "阳", "光", "下", "绽", "放", "的", "红", "玫", "瑰", and combine every two adjacent characters for the first time to obtain multiple character pairs, namely ("一朵"), ("朵在"), ("在阳"), ("阳光"), ("光下"), ("下绽"), ("绽放"), ("放的"), ("的红"), ("红玫"), ("玫瑰"). Then the electronic device can count the frequency of each character pair appearing in the preset word set. Then, the electronic device can determine the first character pair with the highest frequency of appearance in the preset word set from the character pairs obtained in the first combination, that is, the frequency of ("玫瑰") is the largest, and the first character pair is "玫瑰", which can be regarded as a new character. At this time, all the current characters are updated to "一", "朵", "在", "阳", "光", "下", "绽", "放", "的", "红", "玫瑰". Then the electronic device can combine every two adjacent characters among all the current characters for the second time to obtain multiple character pairs, namely ("一朵"), ("朵在"), ("在阳"), ("阳光"), ("光下"), ("下绽"), ("绽放"), ("放的"), ("的红"), ("红玫瑰"). Then the electronic device can count the frequency of the new character pair ("红玫瑰") appearing in the preset word set. Then, the electronic device can determine the first character pair with the highest frequency of appearance in the preset word set from the character pairs obtained in the second combination, that is, the frequency of ("阳光") is the largest, and the first character pair is "阳光", which can be regarded as a new character. At this time, all the current characters are updated to "一", "朵", "在", "阳光", "下", "绽", "放", "的", "红", "玫瑰". Then the electronic device can combine every two adjacent characters among all the current characters for the third time to obtain multiple character pairs, namely ("一朵"), ("朵在"), ("在阳光"), ("阳光下"), ("下绽"), ("绽放"), ("放的"), ("的红"), ("红玫瑰"). Then the electronic device can count the frequencies of the new character pairs ("在阳光") and ("阳光下") appearing in the preset word set. Then, the electronic device can determine the first character pair with the highest frequency of appearance in the preset word set from the character pairs obtained in the third combination, that is, the frequency of ("绽放") is the largest, and the first character pair is "阳光", which can be regarded as a new character.At this time, all the current characters are updated to "a", "flower", "in", "the", "sunshine", "blooming", "of", "red", "rose". Then, the electronic device can combine every two adjacent characters among all the current characters for the 4th time to obtain multiple character pairs, namely ("a flower"), ("flower in"), ("in the sunshine"), ("in the sunshine"), ("blooming of"), ("of red"), ("red rose"). Then, the electronic device can count the frequencies of the new character pairs ("blooming of") and ("of red") in the preset word set. Then, the electronic device can determine the first character pair with the highest frequency in the preset word set from the character pairs obtained in the 4th combination, that is, the frequency of ("a flower") is the highest, and the first character pair is "a flower", which can be regarded as a new character. At this time, all the current characters are updated to "a flower", "in", "the", "sunshine", "blooming", "of", "red", "rose". Then, the electronic device can combine every two adjacent characters among all the current characters for the 5th time to obtain multiple character pairs, namely ("a flower in"), ("in the sunshine"), ("in the sunshine"), ("blooming of"), ("of red"), ("red rose"). Then, the electronic device can count the frequency of the new character pair ("a flower in") in the preset word set. Then, the electronic device can determine the first character pair with the highest frequency in the preset word set from the character pairs obtained in the 5th combination, that is, the frequency of ("red rose") is the highest, and the first character pair is "red rose", which can be regarded as a new character. At this time, all the current characters are updated to "a flower", "in", "the", "sunshine", "blooming", "of", "red rose". Then, the electronic device can combine every two adjacent characters among all the current characters for the 6th time to obtain multiple character pairs, namely ("a flower in"), ("in the sunshine"), ("in the sunshine"), ("blooming of"), ("of red rose"). Then, the electronic device can count the frequency of the new character pair ("of red rose") in the preset word set. Then, the electronic device can determine the first character pair with the highest frequency in the preset word set from the character pairs obtained in the 6th combination, that is, the frequency of ("in the sunshine") is the highest, and the first character pair is "in the sunshine", which can be regarded as a new character. At this time, all the current characters are updated to "a flower", "in", "in the sunshine", "blooming", "of", "red rose". At this time, the combination times reach the preset 6 times. Then, the electronic device can determine all the current characters "a flower", "in", "in the sunshine", "blooming", "of", "red rose" as the above-mentioned at least two word segmentation results, or can determine "a flower", "in the sunshine", "blooming", "red rose" as the above-mentioned at least two word segmentation results.

[0124] The embodiment of the present application provides an image search method based on central image generation and comparison, which helps users to quickly and accurately find target images in complex albums. First, word segmentation is performed based on the user's search terms; second, the results of the search word segmentation are passed into the central image generation model, and images closely related to the search terms are generated from the center point to the surrounding areas according to the word segmentation. Finally, in the process of image generation, it is compared with the pictures in the album, and after the recall, rough sorting, and fine sorting stages, the pictures that the user wants to find are quickly located, which reduces the user's waiting time, improves search efficiency, and can ensure a high accuracy rate in complex albums.

[0125] Each of the above-mentioned method embodiments, or various possible implementation methods in each method embodiment can be executed separately, or any two or more of them can be executed in combination with each other. The specific implementation method can be determined according to actual usage requirements, and the embodiments of the present application do not limit this.

[0126] The image search method provided in the embodiment of the present application can be executed by an image search device. In the embodiment of the present application, an image search device executing the image search method is taken as an example to illustrate the image search device provided in the embodiment of the present application.

[0127] Figure 5 A possible structural diagram of an image search device involved in some embodiments of the present application is shown. As shown in the figure, the image search device 70 may include: a segmentation module 71, a generation module 72 and a determination module 73.

[0128] The segmentation module 71 is used to segment the search term input by the user to obtain at least two segmentation results.

[0129] The generating module 72 is used to generate a first image matching the search term based on at least two word segmentation results segmented by the segmenting module 71 .

[0130] The determination module 73 is used to determine a target image matching the first image from the first image set based on the first image generated by the generation module 72 .

[0131] In one possible implementation, the segmentation module 71 is specifically used to: combine the characters in the search term N times; obtain the at least two word segmentation results based on the character pairs obtained by the Nth combination, where N is an integer greater than 1; wherein, for the first combination, every two adjacent characters in all the characters are combined to obtain at least one character pair; for the i-th combination, determine the first character pair with the highest frequency of occurrence in the preset word set from the character pairs obtained by the i-1-th combination, and combine every two adjacent characters of the remaining characters in all the characters, as well as combine the first character pair with the characters adjacent to the first character pair, i=2,…,N.

[0132] In one possible implementation, the generation module 72 is specifically used to: convert at least two word segmentation results into a numerical vector through an image generation model, and the numerical vector represents the semantic information of the word segmentation results; generate the first image rectangular area based on the numerical vector and the random noise parameter; extract the image features of the n-1th image rectangular area, and generate the nth image rectangular area based on the image features, numerical vector and random noise parameters of the n-1th image rectangular area, where n is an integer greater than 1, and the image features include at least one of the following: image color features, image texture features, image shape features, and image spatial relationship features; when the image composed of all the generated image rectangular areas matches the search term, all the image rectangular areas are synthesized to obtain the first image.

[0133] In one possible implementation, the generation module 72 is also used to: for any image rectangular area among all image rectangular areas, obtain evaluation information based on at least two word segmentation results, any image rectangular area and the any image rectangular area through a judgment model in the image generation model, and the evaluation information represents the authenticity of any image rectangular area; based on the evaluation information, determine whether to regenerate the nth image rectangular area.

[0134] In a possible implementation, the determination module 73 is specifically used to: calculate a first similarity value between the first image and each second image based on a feature vector of the first image, a feature vector of each second image in the first image set, and a preset number of feature values, and determine M images whose first similarity values ​​are greater than or equal to a first threshold value from the first image set, where M is a positive integer; calculate a second similarity value between each third image and each feature value of the first image based on multiple feature values ​​of the first image and multiple feature values ​​of each third image in the M images, to obtain a third similarity value between the first image and each third image, and determine L images whose third similarity values ​​are greater than or equal to the second threshold value from the M images, where L is a positive integer less than or equal to M; calculate a fourth similarity value between each fourth image and the first image based on deep features of each fourth image in the L images and the deep features of the first image, and determine images whose fourth similarity values ​​are greater than or equal to the third threshold value from the L images to obtain a target image.

[0135] An image search device is provided in an embodiment of the present application. Since the image search device can segment the search words to obtain at least two segmentation results that are more conducive to image generation, that is, obtain keywords that can generate images, and then generate corresponding images based on these keywords, that is, the first image that matches the search words, so that the image search device can compare the first image with the images in the first image set to find a target image that matches the first image, that is, the target image found is an image that is identical to or extremely similar to the image generated by the search words input by the user, thereby avoiding the situation where the corresponding picture cannot be matched directly according to the search words input by the user, and the user does not need to search again. In this way, the image search device can accurately find the required pictures, thereby improving the accuracy and efficiency of the image search device in finding pictures.

[0136] The image search device in the embodiment of the present application can be an electronic device, or a component in the electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal, or other devices other than a terminal. Exemplarily, the electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, a vehicle-mounted electronic device, a mobile Internet device (Mobile Internet Device, MID), an augmented reality (augmented reality, AR) / virtual reality (virtual reality, VR) device, a robot, a wearable device, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook or a personal digital assistant (personal digital assistant, PDA), etc. It can also be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personal computer, PC), a television (television, TV), a teller machine or a self-service machine, etc., and the embodiment of the present application is not specifically limited.

[0137] The image search device in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0138] The image search device provided in the embodiment of the present application can implement each process implemented in the above method embodiment, and will not be described again here to avoid repetition.

[0139] Alternatively, if Figure 6As shown, an embodiment of the present application also provides an electronic device 1000, including a processor 1001 and a memory 1002, wherein the memory 1002 stores programs or instructions that can be executed on the processor 1001, and when the program or instructions are executed by the processor 1001, the various steps of the above-mentioned image search method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, they are not described here.

[0140] It should be noted that the electronic devices in the embodiments of the present application include the mobile electronic devices and non-mobile electronic devices mentioned above.

[0141] Figure 7 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of the present application.

[0142] The electronic device 100 includes but is not limited to components such as a radio frequency unit 101, a network module 102, an audio output unit 103, an input unit 104, a sensor 105, a display unit 106, a user input unit 107, an interface unit 108, a memory 109, and a processor 110.

[0143] Those skilled in the art will appreciate that the electronic device 100 may also include a power source (such as a battery) for supplying power to various components, and the power source may be logically connected to the processor 110 through a power management system, thereby implementing functions such as managing charging, discharging, and power consumption management through the power management system. Figure 7 The electronic device structure shown in the figure does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently, which will not be described in detail here.

[0144] The processor 110 is used to segment the search term input by the user to obtain at least two segmentation results.

[0145] The processor 110 is further configured to generate a first image matching the search term based on at least two word segmentation results.

[0146] The processor 110 is further configured to determine, based on the first image, a target image matching the first image from the first image set.

[0147] An embodiment of the present application provides an electronic device, which can segment search words to obtain at least two segmentation results that are more conducive to image generation, that is, obtain keywords that can generate images, and then generate corresponding images based on these keywords, that is, the first image that matches the search words, so that the electronic device can compare the first image with the images in the first image set to find a target image that matches the first image, that is, the target image found is an image that is identical to or extremely similar to the image generated by the search word input by the user, thereby avoiding the situation where the corresponding picture cannot be matched directly based on the search word input by the user, and the user does not need to search again, so that the electronic device can accurately find the required pictures, thereby improving the accuracy and efficiency of the electronic device in finding pictures.

[0148] Optionally, the processor 110 is specifically used to: combine the characters in the search term N times; obtain the at least two word segmentation results based on the character pairs obtained by the Nth combination, where N is an integer greater than 1; wherein, for the 1st combination, every two adjacent characters in all the characters are combined to obtain at least one character pair; for the i-th combination, determine the first character pair with the highest frequency of occurrence in the preset word set from the character pairs obtained by the i-1th combination, and combine every two adjacent characters of the remaining characters in all the characters, as well as combine the first character pair with the characters adjacent to the first character pair, i=2,…,N.

[0149] Optionally, the processor 110 is specifically used to: convert at least two word segmentation results into a numerical vector through an image generation model, and the numerical vector represents the semantic information of the word segmentation results; generate a first image rectangular area based on the numerical vector and random noise parameters; extract image features of the n-1th image rectangular area, and generate an nth image rectangular area based on the image features, numerical vectors and random noise parameters of the n-1th image rectangular area, where n is an integer greater than 1, and the image features include at least one of the following: image color features, image texture features, image shape features, and image spatial relationship features; when an image composed of all generated image rectangular areas matches the search term, all image rectangular areas are synthesized to obtain a first image.

[0150] Optionally, the processor 110 is also used to: for any image rectangular area among all image rectangular areas, obtain evaluation information based on at least two word segmentation results, any image rectangular area and the any image rectangular area through a judgment model in the image generation model, wherein the evaluation information represents the authenticity of any image rectangular area; and determine whether to regenerate any image rectangular area based on the evaluation information.

[0151] Optionally, the processor 110 is specifically used to: calculate a first similarity value between the first image and each second image based on a feature vector of the first image, a feature vector of each second image in the first image set, and a preset number of feature values, and determine M images from the first image set whose first similarity values ​​are greater than or equal to a first threshold, where M is a positive integer; calculate a second similarity value between each third image and each feature value of the first image based on multiple feature values ​​of the first image and multiple feature values ​​of each third image in the M images, to obtain a third similarity value between the first image and each third image, and determine L images from the M images whose third similarity values ​​are greater than or equal to the second threshold, where L is a positive integer less than or equal to M; calculate a fourth similarity value between each fourth image and the first image based on deep features of each fourth image in the L images and the deep features of the first image, and determine images whose fourth similarity values ​​are greater than or equal to the third threshold from the L images to obtain a target image.

[0152] The electronic device provided in the embodiment of the present application can implement each process implemented in the above method embodiment, and can achieve the same technical effect. To avoid repetition, it is not repeated here. The beneficial effects of various implementations in this embodiment can be specifically referred to the beneficial effects of the corresponding implementations in the above method embodiment. To avoid repetition, it is not repeated here.

[0153] It should be understood that in the embodiment of the present application, the input unit 104 may include a graphics processor (Graphics Processing Unit, GPU) 1041 and a microphone 1042, and the graphics processor 1041 processes the image data of a static picture or video obtained by an image capture device (such as a camera) in a video capture mode or an image capture mode. The display unit 106 may include a display panel 1061, and the display panel 1061 may be configured in the form of a liquid crystal display, an organic light emitting diode, etc. The user input unit 107 includes a touch panel 1071 and at least one of other input devices 1072. The touch panel 1071 is also called a touch screen. The touch panel 1071 may include two parts: a touch detection device and a touch controller. Other input devices 1072 may include, but are not limited to, a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, and a joystick, which will not be repeated here.

[0154] The memory 109 can be used to store software programs and various data. The memory 109 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area may store an operating system, an application program or instructions required for at least one function (such as a sound playback function, an image playback function, etc.), etc. In addition, the memory 109 may include a volatile memory or a non-volatile memory, or the memory 109 may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDRSDRAM), an enhanced synchronous dynamic random access memory (ESDRAM), a synchronous link dynamic random access memory (SLDRAM) and a direct memory bus random access memory (DRRAM). The memory 109 in the embodiment of the present application includes but is not limited to these and any other suitable types of memory.

[0155] The processor 110 may include one or more processing units; optionally, the processor 110 integrates an application processor and a modem processor, wherein the application processor mainly processes operations related to an operating system, a user interface, and application programs, and the modem processor mainly processes wireless communication signals, such as a baseband processor. It is understandable that the modem processor may not be integrated into the processor 110.

[0156] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the various processes of the above-mentioned image search method embodiment are implemented and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0157] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.

[0158] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned image search method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0159] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0160] An embodiment of the present application provides a computer program product, which is stored in a storage medium. The program product is executed by at least one processor to implement the various processes of the above-mentioned image search method embodiment and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0161] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0162] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0163] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. An image search method, characterized in that: include: Split the search term entered by the user to obtain at least two segmentation results; Based on the at least two word segmentation results, generating a first image matching the search word; Based on the first image, a target image matching the first image is determined from a first image set.

2. The method according to claim 1, characterized in that The step of segmenting the search term input by the user to obtain at least two segmentation results includes: Combining characters in the search term N times; Based on the character pairs obtained by the Nth combination, obtaining the at least two word segmentation results, where N is an integer greater than 1; Wherein, for the first combination, every two adjacent characters among all the characters are combined to obtain at least one character pair; For the i-th combination, determine the first character pair with the highest frequency of occurrence in the preset word set from the character pairs obtained from the i-1-th combination, and combine every two adjacent characters of the remaining characters in all the characters, as well as combine the first character pair with the characters adjacent to the first character pair, i=2,…,N.

3. The method according to claim 1 or 2, characterized in that: The step of generating a first image matching the search term based on the at least two word segmentation results includes: The at least two word segmentation results are converted into numerical vectors through an image generation model, wherein the numerical vectors represent semantic information of the word segmentation results; Based on the numerical vector and the random noise parameter, generate a first image rectangular area; Extracting image features of the n-1th image rectangular region, and generating an nth image rectangular region based on the image features of the n-1th image rectangular region, the numerical vector and the random noise parameter, where n is an integer greater than 1, and the image features include at least one of the following: color features of the image, texture features of the image, shape features of the image, and spatial relationship features of the image; In the case that the image formed by all the generated image rectangular regions matches the search word, all the image rectangular regions are synthesized to obtain the first image.

4. The method according to claim 3, characterized in that The method further comprises: For any image rectangular region among all the image rectangular regions, obtaining evaluation information through a judgment model in the image generation model based on the at least two word segmentation results, the any image rectangular region, and a previous image rectangular region of the any image rectangular region, wherein the evaluation information represents a degree of matching between the any image rectangular region and the at least two word segmentation results; Based on the evaluation information, it is determined whether to regenerate any one of the image rectangular regions.

5. The method according to claim 1, characterized in that The determining, based on the first image, a target image matching the first image from a first image set includes: Based on the feature vector of the first image, the feature vector of each second image in the first image set, and the number of preset feature values, calculate a first similarity value between the first image and each second image, and determine M images whose first similarity values ​​are greater than or equal to a first threshold from the first image set, where M is a positive integer; Based on multiple feature values ​​of the first image and multiple feature values ​​of each third image in the M images, calculate a second similarity value between each third image and each feature value of the first image to obtain a third similarity value between the first image and each third image, and determine L images whose third similarity values ​​are greater than or equal to a second threshold from the M images, where L is a positive integer less than or equal to M; Based on the deep features of each fourth image in the L images and the deep features of the first image, a fourth similarity value between each fourth image and the first image is calculated, and an image whose fourth similarity value is greater than or equal to a third threshold is determined from the L images to obtain the target image.

6. A picture search device, characterized in that: include: Segmentation module, generation module and determination module; The segmentation module is used to segment the search term input by the user to obtain at least two segmentation results; The generating module is configured to generate a first image matching the search term based on the at least two word segmentation results segmented by the segmentation module; The determination module is used to determine a target image matching the first image from a first image set based on the first image generated by the generation module.

7. The device according to claim 6, characterized in that The segmentation module is specifically used for: Combining characters in the search term N times; Based on the character pairs obtained by the Nth combination, obtaining the at least two word segmentation results, where N is an integer greater than 1; Wherein, for the first combination, every two adjacent characters among all the characters are combined to obtain at least one character pair; For the i-th combination, determine the first character pair with the highest frequency of occurrence in the preset word set from the character pairs obtained from the i-1-th combination, and combine every two adjacent characters of the remaining characters in all the characters, as well as combine the first character pair with the characters adjacent to the first character pair, i=2,…,N.

8. The device according to claim 6 or 7, characterized in that The generating module is specifically used for: The at least two word segmentation results are converted into numerical vectors through an image generation model, wherein the numerical vectors represent semantic information of the word segmentation results; Based on the numerical vector and the random noise parameter, generate a first image rectangular area; Extracting image features of the n-1th image rectangular region, and generating an nth image rectangular region based on the image features of the n-1th image rectangular region, the numerical vector and the random noise parameter, where n is an integer greater than 1, and the image features include at least one of the following: color features of the image, texture features of the image, shape features of the image, and spatial relationship features of the image; In the case that the image formed by all the generated image rectangular regions matches the search word, all the image rectangular regions are synthesized to obtain the first image.

9. The device according to claim 8, characterized in that The generating module is further used for: For any image rectangular region among all the image rectangular regions, obtaining evaluation information through a judgment model in the image generation model based on the at least two word segmentation results, the any image rectangular region, and a previous image rectangular region of the any image rectangular region, wherein the evaluation information represents a degree of matching between the any image rectangular region and the at least two word segmentation results; Based on the evaluation information, it is determined whether to regenerate any one of the image rectangular regions.

10. The device according to claim 6, characterized in that The determining module is specifically used for: Based on the feature vector of the first image, the feature vector of each second image in the first image set, and the number of preset feature values, calculate a first similarity value between the first image and each second image, and determine M images whose first similarity values ​​are greater than or equal to a first threshold from the first image set, where M is a positive integer; Based on multiple feature values ​​of the first image and multiple feature values ​​of each third image in the M images, calculate a second similarity value between each third image and each feature value of the first image to obtain a third similarity value between the first image and each third image, and determine L images whose third similarity values ​​are greater than or equal to a second threshold from the M images, where L is a positive integer less than or equal to M; Based on the deep features of each fourth image in the L images and the deep features of the first image, a fourth similarity value between each fourth image and the first image is calculated, and an image whose fourth similarity value is greater than or equal to a third threshold is determined from the L images to obtain the target image.