Image retrieval method and device, electronic equipment, storage medium and program product

Abstract

The embodiment of the invention discloses an image retrieval method and device, electronic equipment, a storage medium and a program product, and the method comprises the steps: selecting at least one hash function corresponding to a target image from a hash function set through employing a hash function selection network; generating a retrieval prefix based on the at least one hash function; performing fitting calculation on the image features of the target image based on the hash function set by using a hash calculation network to obtain a hash value; the retrieval prefix and the hash value are spliced, and a retrieval index is obtained; and performing image retrieval based on the retrieval index. According to the embodiment of the invention, the retrieval efficiency can be greatly improved on the premise of ensuring the image retrieval precision.

Description

Technical Field

[0001] The present disclosure relates to artificial intelligence technology, and in particular to an image retrieval method, device, electronic device, storage medium, and program product. Background Art

[0002] Traditional image retrieval involves manually annotating images with text and then using keywords to retrieve the images. This method, which provides retrieval results based on the degree of character matching in the image description, is very time-consuming and easily influenced by user subjectivity, resulting in inaccurate text annotations or keywords, and thus low retrieval accuracy.

[0003] Given a query image, quickly identifying a certain number of images with similar content from a large image database using image features is a popular research topic. Related technologies employ image retrieval methods based on locality-sensitive hashing (LSH). These methods perform hash calculations on all hash functions in a set during image retrieval, resulting in large amounts of computation and slow retrieval speeds. Summary of the Invention

[0004] The embodiments of the present disclosure provide an image retrieval method, apparatus, electronic device, storage medium, and program product to solve the above-mentioned problems.

[0005] According to a first aspect of the present disclosure, an image retrieval method is provided, comprising: Selecting at least one hash function corresponding to the target image from a set of hash functions using a hash function selection network; generating a search prefix based on the at least one hash function; Using a hash calculation network, performing fitting calculation on the image features of the target image based on the hash function set to obtain a hash value, wherein the hash function selection network and the hash calculation network are jointly trained based on sample images; Concatenate the search prefix and the hash value to obtain a search index; Image retrieval is performed based on the retrieval index.

[0006] In some embodiments of the present disclosure, the step of selecting at least one hash function corresponding to the target image from a set of hash functions using a hash function selection network includes: determining, based on the hash function selection network, characteristics of each hash function in the set of hash functions; The at least one hash function is selected from the set of hash functions based on a difference between characteristics of the hash functions and image characteristics of the target image.

[0007] In some embodiments of the present disclosure, generating a search prefix based on the at least one hash function includes: Determine a value filling position of each hash function in the hash function set in the search prefix; A first preset value is filled in the value filling position corresponding to the at least one hash function, and a second preset value is filled in the value filling positions corresponding to the remaining hash functions in the hash function set except the at least one hash function, to obtain the search prefix.

[0008] In some embodiments of the present disclosure, the using a hash computing network to perform fitting calculations on the image features of the target image based on the hash function set includes: Using a twin neural network, performing a hash calculation based on the image features of the target image and the hash function set to obtain a first hash vector and a second hash vector; Projecting the first hash vector and the second hash vector respectively to obtain a first hash function and a second hash function; A fitting calculation is performed based on the hash function set, the first hash function, and the second hash function to determine the hash value.

[0009] In some embodiments of the present disclosure, all hash functions in the hash function set are locality-sensitive hash functions.

[0010] According to a second aspect of the present disclosure, an image retrieval device is provided, comprising: A hash function selection module, configured to select at least one hash function corresponding to a target image from a set of hash functions using a hash function selection network; a search prefix obtaining module, configured to generate a search prefix based on the at least one hash function; a hash value acquisition module, configured to utilize a hash calculation network to perform fitting calculation on the image features of the target image based on the hash function set, wherein the hash function selection network and the hash calculation network are jointly trained based on sample images; A search index acquisition module, configured to concatenate the search prefix and the hash value to obtain a search index; An image retrieval module is used to perform image retrieval based on the retrieval index.

[0011] In some embodiments of the present disclosure, the hash function selection module includes: a feature determination unit, configured to determine a feature of each hash function in the set of hash functions based on the hash function selection network; A hash function selection unit is configured to select the at least one hash function from the hash function set based on a difference between features of the hash functions and image features of the target image.

[0012] In some embodiments of the present disclosure, the search prefix acquisition module includes: a value filling position determining unit, configured to determine a value filling position of each hash function in the hash function set in the search prefix; A value filling unit is used to fill a first preset value in the value filling position corresponding to the at least one hash function, and fill a second preset value in the value filling position corresponding to the remaining hash functions in the hash function set except the at least one hash function, to obtain the search prefix.

[0013] In some embodiments of the present disclosure, the hash value acquisition module includes: a hash vector acquisition unit, configured to perform a hash calculation based on the image features of the target image and the at least one hash function using a twin neural network to obtain a first hash vector and a second hash vector; a projection unit, configured to project the first hash vector and the second hash vector respectively to obtain a first hash function and a second hash function; A hash value determining unit is configured to perform a fitting calculation based on the hash function set, the first hash function, and the second hash function to determine the hash value.

[0014] In some embodiments of the present disclosure, all hash functions in the hash function set are locality-sensitive hash functions.

[0015] According to a third aspect of the present disclosure, an electronic device is provided, including: a memory for storing a computer program product; The processor is configured to execute the computer program product stored in the memory, and when the computer program product is executed, the method described in the first aspect above is implemented.

[0016] According to a fourth aspect of the embodiments of the present disclosure, a computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the method described in the first aspect is implemented.

[0017] According to a fifth aspect of the embodiments of the present disclosure, a computer program product is provided, comprising computer program instructions, which, when executed by a processor, enable the processor to execute the method described in the first aspect.

[0018] The image retrieval method, device, electronic device, storage medium and program product of the embodiments of the present disclosure, since performing hash calculations on all hash functions in a hash function set during image retrieval affects the retrieval efficiency, some hash functions that match the target image are selected from the hash function set to generate a retrieval prefix, and the hash function set is fitted according to the image features of the target image to obtain a hash value, and then the retrieval prefix and the hash value are combined to obtain a retrieval index. By effectively utilizing the retrieval prefix, the image retrieval method can greatly improve the retrieval efficiency while ensuring the image retrieval accuracy.

[0019] The technical solution of the present disclosure is further described in detail below through the accompanying drawings and examples. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0021] The present disclosure can be more clearly understood from the following detailed description with reference to the accompanying drawings, in which: Figure 1 Schematic diagram of the process of an image retrieval method in one embodiment of the present disclosure; Figure 2 is a flow chart of step S1 in some embodiments of the present disclosure; Figure 3 is a flow chart of step S2 in some embodiments of the present disclosure; Figure 4 is a flow chart of step S3 in some embodiments of the present disclosure; Figure 5 This is a diagram showing the calculation principle of a hash value in an example of the present disclosure; Figure 6 is a structural block diagram of an image retrieval device in some embodiments of the present disclosure; Figure 7 is a structural block diagram of the hash function selection module 100 in some embodiments of the present disclosure; Figure 8 is a structural block diagram of the search prefix acquisition module 200 in some embodiments of the present disclosure; Figure 9 is a structural block diagram of the hash value acquisition module 300 in some embodiments of the present disclosure; Figure 10 This is a structural block diagram of an electronic device in one embodiment of the present disclosure. DETAILED DESCRIPTION

[0022] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that unless otherwise specifically stated, the relative arrangement of components and steps, numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present disclosure.

[0023] Those skilled in the art will understand that the terms "first" and "second" in the embodiments of the present disclosure are only used to distinguish different steps, devices or modules, and do not represent any specific technical meanings, nor do they indicate a necessary logical order between them.

[0024] It should also be understood that in the embodiments of the present disclosure, “a plurality of” may refer to two or more than two, and “at least one” may refer to one, two, or more than two.

[0025] It should also be understood that any component, data or structure mentioned in the embodiments of the present disclosure can generally be understood as one or more, unless explicitly limited or otherwise indicated in the context.

[0026] In addition, the term "and / or" in this disclosure is merely a description of the association relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone. In addition, the character " / " in this disclosure generally indicates that the related objects are in an "or" relationship.

[0027] It should also be understood that the description of the various embodiments in this disclosure focuses on the differences between the various embodiments, and the same or similar aspects thereof can be referenced with each other. For the sake of brevity, they will not be described one by one.

[0028] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the present disclosure, its application, or uses.

[0029] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered part of the specification.

[0030] It should be noted that like reference numerals and letters refer to like items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0031] The embodiments of the present disclosure can be applied to electronic devices such as terminal devices, computer systems, and servers, and can operate in conjunction with numerous other general-purpose or specialized computing system environments or configurations. Examples of well-known terminal devices, computing systems, environments, and / or configurations suitable for use with terminal devices, computer systems, servers, and other electronic devices include, but are not limited to, personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, microprocessor-based systems, set-top boxes, programmable consumer electronics, network personal computers, minicomputer systems, mainframe computer systems, and distributed cloud computing technology environments including any of the above systems, among others.

[0032] Electronic devices such as terminal devices, computer systems, and servers can be described in the general context of computer system-executable instructions (such as program modules) executed by the computer system. Generally, program modules can include routines, programs, object programs, components, logic, data structures, and the like that perform specific tasks or implement specific abstract data types. Computer systems / servers can be implemented in distributed cloud computing environments, where tasks are performed by remote processing devices linked via a communications network. In distributed cloud computing environments, program modules can be located on local or remote computer system storage media, including storage devices.

[0033] Figure 1 FIG. 1 is a flow chart of an image retrieval method in one embodiment of the present disclosure. Figure 1 As shown, the image retrieval method includes the following steps: S1: Using a hash function selection network, select at least one hash function corresponding to a target image from a hash function set.

[0034] Sets a hash function set for image retrieval. The hash function set includes multiple hash functions, and each hash function has different characteristics when applied to image retrieval.

[0035] The target image is input into a pre-trained hash function selection network, and the hash function selection network is used to select at least one hash function corresponding to the target image from a set of hash functions according to a function selection condition corresponding to the hash function selection network.

[0036] S2: Generate a search prefix based on at least one hash function.

[0037] All hash functions in the hash function set are sorted, and each hash function is assigned a corresponding identifier. According to the position of at least one selected hash function in the hash function set, a search prefix having a hash function sorting relationship is generated.

[0038] In an example of the present disclosure, the hash function set includes n hash functions, and the identifiers of the n functions are respectively h 1, h 2,..., h n, where n is an integer greater than 2. At least one hash function selected from the hash function set includes h i, h j and h k, where 1 ≤ i < j < k ≤ n. A retrieval prefix such as Z,..., h i,..., h j,..., h k,..., Z can be generated according to the positions of the functions h i, h j, h k in the hash function set. Here, Z is a preset value indicating that the function is not selected.

[0039] S3: Using the hash calculation network, perform fitting calculation on the image features of the target image based on the hash function set to obtain a hash value. The hash function selection network and the hash calculation network are jointly trained based on the sample images.

[0040] After extracting the image features of the target image using the hash calculation network, according to the function fitting calculation method corresponding to the hash calculation network, fit the hash function set. Then, perform hash calculation on the image features of the reference image to obtain a hash value.

[0041] Among them, the hash function selection network and the hash calculation network are jointly trained based on the sample images. The process of jointly training the hash function selection network and the hash calculation network may include: using the initial hash function selection network, selecting at least one hash function corresponding to the sample image from the hash function set, and generating a retrieval prefix according to the at least one selected hash function; using the initial hash calculation network, perform fitting calculation on the image features of the sample image based on the hash function set to obtain a hash value, splice the retrieval prefix and the hash value and perform retrieval in the image library, and adjust the parameters of the initial hash function selection network and the initial hash calculation network according to the retrieval result to obtain the hash function selection network and the hash calculation network.

[0042] S4: Splice the retrieval prefix and the hash value to obtain a retrieval index.

[0043] S5: Perform image retrieval based on the retrieval index.

[0044] According to the retrieval index, perform retrieval in the network or in the specified image library to obtain multiple images similar to the target image features.

[0045] In this embodiment, since the retrieval efficiency will be affected if hash calculation is performed on all hash functions in the hash function set during image retrieval, some hash functions that match the target image are selected from the hash function set to generate a retrieval prefix, and the hash function set is fitted according to the image features of the target image to obtain a hash value, and then the retrieval prefix and the hash value are combined to obtain a retrieval index. By effectively utilizing the image retrieval prefix, the retrieval efficiency can be greatly improved while ensuring the image retrieval accuracy.

[0046] Figure 2 Schematic diagram of the process of step S1 in some embodiments of the present disclosure. Figure 2 As shown, step S1 includes the following steps: S1-1: Based on the hash function selection network, determine the characteristics of each hash function in the hash function set.

[0047] The function feature extraction network in the hash function selection network is used to extract the features of each hash function in the hash function set.

[0048] S1-2: Select at least one hash function from a set of hash functions based on differences between features of each hash function and image features of the target image.

[0049] The difference between the characteristics of each hash function and the image characteristics of the target image is determined by using a comparison method for determining the difference between the characteristics of each hash function and the image characteristics of the target image in the hash function selection network.

[0050] According to the differences between the features of each hash function and the image features of the target image, a preset number of hash functions are selected from the hash function set in descending order of the differences to obtain at least one hash function.

[0051] In this embodiment, since searching the target image can obtain better retrieval results when the difference between the characteristics of the hash function and the image characteristics of the target image is large, the hash function selection network is used to select at least one hash function with a large difference in image characteristics from the target image from the hash function set, which helps to improve the retrieval accuracy.

[0052] Figure 3 Schematic diagram of the process of step S2 in some embodiments of the present disclosure. Figure 3 As shown, step S2 includes the following steps: S2-1: Determine the value filling position of each hash function in the hash function set in the search prefix.

[0053] A blank search prefix of a length corresponding to the number of hash functions in the hash function set can be generated, and each blank position in the blank search prefix is ​​filled with a value indicating whether a hash function in the hash function set is selected.

[0054] S2-2: Fill a first preset value in a value filling position corresponding to at least one hash function, and fill a second preset value in a value filling position corresponding to the remaining hash functions in the hash function set except for the at least one hash function, to obtain a search prefix.

[0055] At least one value filling position corresponding to a hash function in a blank search prefix is ​​filled with a first preset value representing a selected function, and the remaining positions of the blank search prefix are filled with a second preset value representing a function that violates the selected function.

[0056] In some examples of the present disclosure, the hash function set includes 10 hash functions, and at least one hash function selected by the hash function includes the third, fifth, and eighth hash functions in the hash function set. The search prefix may be: {0,0,1,0,1,0,0,1,0,0}. The third, fifth, and eighth hash functions in the hash function set are selected, and the third, fifth, and eighth value filling positions of the hash prefix are filled with the value 1. The remaining positions corresponding to the hash functions are not selected and are filled with the value 0.

[0057] In this embodiment, since the use of hash prefixes can improve retrieval efficiency, a hash prefix consisting of a numerical queue is generated based on whether each hash function in the hash function set is selected by the hash function, which represents whether the corresponding hash function is selected. This can reduce the data storage capacity of the retrieval prefix while improving retrieval efficiency.

[0058] Figure 4 Schematic diagram of the process of step S3 in some embodiments of the present disclosure. Figure 4 As shown, step S3 includes the following steps: S3-1: Using the twin neural network, hash calculation is performed based on the image features of the target image and the hash function set to obtain a first hash vector and a second hash vector.

[0059] Figure 5 This is a diagram showing the calculation principle of the hash value in an example of the present disclosure. Figure 5 As shown, the stack trace encoders (stack trace encoder 1 and stack trace encoder 2) corresponding to the stack pair (stack 1 and stack 2) of the twin neural network are encoded to generate the corresponding feature map (feature Figure 1 and features Figure 2), and then provides the feature map to the corresponding multi-network layer (multi-network layer 1 and multi-network layer 2). In addition, the hash selection network provides a set of hash functions to the multi-network layer of the twin neural network, so that the multi-network layer calculates a first hash vector and a second hash vector based on the hash function set and the feature map. The first hash vector and the second hash vector are multiple concatenated binary hash codes, each of which is encoded as b bits in the Hamming space, where b is an integer greater than 1.

[0060] S3-2: Project the first hash vector and the second hash vector respectively to obtain a first hash function and a second hash function.

[0061] The first hash vector and the second hash vector are projected respectively using a preset projection function to obtain a first hash function and a second hash function.

[0062] S3-3: Perform fitting calculation based on the hash function set, the first hash function, and the second hash function to determine a hash value.

[0063] Since hash vectors are in the Hamming space, that is, hash vectors are restricted to binary values, {0, 1} or {-1, 1}, computing the collision probability between two random projected hash functions of the same order h is equivalent to computing the Hamming similarity between two hash vectors H(si) and H(sj), where H(si) and H(sj) represent the first hash vector and the second hash vector.

[0064] Considering that the hash function encodes b bits (b > 1), meaning it is not limited to a single projection but rather b coordinates, we need to generalize the property of b > 1. We define the generalized Hamming distance between two hash vectors H(si) and H(sj) as the number of k-order different projection hash functions. We perform a fitting calculation based on the hash function set, the first hash function, and the second hash function, and then obtain the hash value based on the fitting calculation.

[0065] In this embodiment, since the twin neural network is more robust to the problem of category imbalance and can learn to estimate the distance between two inputs based on semantic similarity, the twin neural network is used to perform hash calculation and projection based on image features and a set of hash functions to obtain two hash functions. The distance between the two hash functions can represent the similarity between the two hash functions, and then a hash value that can reasonably represent the image features is obtained by hash function fitting, which helps to improve the image retrieval accuracy.

[0066] In some embodiments of the present disclosure, all hash functions in the hash function set are locality-sensitive hash functions.

[0067] In this embodiment, since the local sensitive hash function has the characteristic that if two objects are similar in the original data space, they will also have a high degree of similarity after being converted by the hash function respectively, the use of the local sensitive hash function for image retrieval can greatly improve the image retrieval accuracy.

[0068] Figure 6 FIG is a structural block diagram of an image retrieval device in some embodiments of the present disclosure. Figure 6 As shown, the image retrieval device includes: A hash function selection module 100 is configured to select at least one hash function corresponding to a target image from a set of hash functions using a hash function selection network; A search prefix acquisition module 200, configured to generate a search prefix based on at least one hash function; A hash value acquisition module 300 is configured to use a hash calculation network to perform fitting calculations on image features of a target image based on a set of hash functions, wherein the hash function selection network and the hash calculation network are jointly trained based on sample images; The search index acquisition module 400 is used to concatenate the search prefix and the hash value to obtain the search index; The image retrieval module 500 is used to perform image retrieval based on the retrieval index.

[0069] Figure 7 FIG. 1 is a block diagram of the structure of the hash function selection module 100 in some embodiments of the present disclosure. Figure 7 As shown, the hash function selection module 100 includes: a feature determination unit 110 for determining a feature of each hash function in a set of hash functions based on a hash function selection network; The hash function selection unit 120 is configured to select at least one hash function from the hash function set based on the difference between the characteristics of each hash function and the image characteristics of the target image.

[0070] Figure 8 FIG. 2 is a structural block diagram of the search prefix acquisition module 200 in some embodiments of the present disclosure. Figure 8 As shown, the search prefix acquisition module 200 includes: A value filling position determining unit 210 is used to determine a value filling position of each hash function in the hash function set in the search prefix; The value filling unit 220 is used to fill the first preset value in the value filling position corresponding to at least one hash function, and fill the second preset value in the value filling position corresponding to the remaining hash functions in the hash function set except at least one hash function, to obtain a search prefix.

[0071] Figure 9is a structural block diagram of the hash value acquisition module 300 in some embodiments of the present disclosure. Figure 9 As shown, the hash value acquisition module 300 includes: A hash vector acquisition unit 310 is configured to use a Siamese neural network to perform a hash calculation based on image features of the target image and at least one hash function to obtain a first hash vector and a second hash vector; A projection unit 320 is configured to project the first hash vector and the second hash vector respectively to obtain a first hash function and a second hash function; The hash value determining unit 330 is configured to perform fitting calculation based on the hash function set, the first hash function, and the second hash function to determine a hash value.

[0072] In some embodiments of the present disclosure, all hash functions in the hash function set are locality-sensitive hash functions.

[0073] It should be noted that the specific implementation of the image retrieval device of the embodiment of the present disclosure is similar to the specific implementation of the image retrieval method of the embodiment of the present disclosure, and the technical effect of the image retrieval device of the embodiment of the present disclosure is similar to the technical effect of the image retrieval method of the embodiment of the present disclosure. Please refer to the description of the image retrieval method part for details. In order to reduce redundancy, it will not be repeated.

[0074] In addition, an embodiment of the present disclosure further provides an electronic device, including: memory for storing computer programs; The processor is configured to execute the computer program stored in the memory, and when the computer program is executed, the image retrieval method described in any one of the above embodiments of the present disclosure is implemented.

[0075] Below, reference Figure 10 To describe the electronic device according to the embodiment of the present disclosure. Figure 10 As shown, the electronic device includes one or more processors and memory.

[0076] The processor may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device to perform desired functions.

[0077] The memory may store one or more computer program products. The memory may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory. The non-volatile memory may include, for example, read-only memory (ROM), a hard disk, flash memory, etc. One or more computer program products may be stored on the computer-readable storage medium, and the processor may execute the computer program products to implement the image retrieval methods of the various embodiments of the present disclosure described above and / or other desired functions.

[0078] In one example, the electronic device may further include an input device and an output device, and these components are interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0079] In addition, the input device may also include, for example, a keyboard, a mouse, and the like.

[0080] The output device can output various information to the outside, including determined distance information, direction information, etc. The output device can include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto, and the like.

[0081] Of course, to simplify, Figure 10 Only some of the components related to the present disclosure in the electronic device are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, the electronic device may further include any other appropriate components according to specific application scenarios.

[0082] In addition to the above-mentioned methods and devices, an embodiment of the present disclosure may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to perform the steps of the image retrieval method according to various embodiments of the present disclosure described in the above part of this specification.

[0083] The computer program product may be written in any combination of one or more programming languages ​​to implement the operations of the disclosed embodiments, including object-oriented programming languages ​​such as Java, C++, and conventional procedural programming languages ​​such as C or similar programming languages. The program code may be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0084] In addition, an embodiment of the present disclosure may also be a computer-readable storage medium having computer program instructions stored thereon, which, when executed by a processor, enable the processor to execute the steps of the image retrieval method according to various embodiments of the present disclosure described in the above part of this specification.

[0085] The computer-readable storage medium may be any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may include, for example, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0086] The basic principles of the present disclosure have been described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, and effects mentioned in this disclosure are merely illustrative and not restrictive, and should not be construed as necessarily possessed by each embodiment of the present disclosure. Furthermore, the specific details disclosed above are provided for illustrative purposes and to facilitate understanding, rather than as limitations. These details do not limit the present disclosure to necessarily being implemented using these specific details.

[0087] Each embodiment in this specification is described in a progressive manner, with each embodiment focusing on its differences from the other embodiments. References to the same or similar parts between the various embodiments are sufficient. For system embodiments, since they largely correspond to method embodiments, their description is relatively simple. For relevant parts, references to the description of the method embodiments are sufficient.

[0088] The block diagrams of the devices, devices, equipment, and systems involved in this disclosure are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As will be appreciated by those skilled in the art, these devices, devices, equipment, and systems can be connected, arranged, or configured in any manner. Words such as "include," "comprise," "have," and the like are open-ended words, meaning "including but not limited to," and can be used interchangeably therewith. The words "or" and "and" used herein refer to the words "and / or" and can be used interchangeably therewith, unless the context clearly indicates otherwise. The word "such as" used herein refers to the phrase "such as but not limited to," and can be used interchangeably therewith.

[0089] The methods and apparatus of the present disclosure may be implemented in many ways. For example, the methods and apparatus of the present disclosure may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above order of steps for the method is for illustration only, and the steps of the method of the present disclosure are not limited to the order specifically described above unless otherwise specified. In addition, in some embodiments, the present disclosure may also be implemented as programs recorded in a recording medium, which include machine-readable instructions for implementing the methods according to the present disclosure. Thus, the present disclosure also covers recording media that store programs for executing the methods according to the present disclosure.

[0090] It should also be noted that in the apparatus, device, and method of the present disclosure, each component or each step can be decomposed and / or recombined. Such decomposition and / or recombination should be regarded as equivalent solutions of the present disclosure.

[0091] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present disclosure. Therefore, the present disclosure is not intended to be limited to the aspects shown herein, but rather to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0092] The above description has been provided for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present disclosure to the forms disclosed herein. Although a number of example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

Claims

1. An image retrieval method, characterized in that: include: Using a hash function selection network, selecting at least one hash function corresponding to the target image from a set of hash functions; Based on the at least one hash function, generating a search prefix; Using a hash calculation network, performing fitting calculation on the image features of the target image based on the hash function set to obtain a hash value, wherein the hash function selection network and the hash calculation network are jointly trained based on a sample image; Concatenate the search prefix and the hash value to obtain a search index; Image retrieval is performed based on the retrieval index.

2. The method according to claim 1, characterized in that The method of selecting at least one hash function corresponding to the target image from a set of hash functions by using a hash function selection network includes: Determining a characteristic of each hash function in the set of hash functions based on the hash function selection network; The at least one hash function is selected from the set of hash functions based on a difference between the characteristics of the hash functions and the image characteristics of the target image.

3. The method according to claim 1 or 2, characterized in that: The step of generating a search prefix based on the at least one hash function comprises: Determine a value filling position of each hash function in the hash function set in the search prefix; A first preset value is filled in the value filling position corresponding to the at least one hash function, and a second preset value is filled in the value filling position corresponding to the remaining hash functions in the hash function set except the at least one hash function, to obtain the search prefix.

4. The method according to claim 1 or 2, characterized in that: The using a hash computing network to perform fitting calculation on the image features of the target image based on the hash function set includes: Using a twin neural network, performing hash calculation based on the image features of the target image and the hash function set to obtain a first hash vector and a second hash vector; Projecting the first hash vector and the second hash vector respectively to obtain a first hash function and a second hash function; A fitting calculation is performed based on the hash function set, the first hash function, and the second hash function to determine the hash value.

5. The method according to claim 1 or 2, characterized in that: The hash functions in the hash function set are all locality-sensitive hash functions.

6. An image retrieval device, characterized in that: include: A hash function selection module, used to select at least one hash function corresponding to the target image from the hash function set using a hash function selection network; A search prefix acquisition module, used to generate a search prefix based on the at least one Hash function; A hash value acquisition module, used for performing fitting calculation on the image features of the target image based on the hash function set using a hash calculation network, wherein the hash function selection network and the hash calculation network are obtained by joint training based on sample images; A search index acquisition module, used to concatenate the search prefix and the hash value to obtain a search index; The image retrieval module is used to perform image retrieval based on the retrieval index.

7. The device according to claim 6, characterized in that The hash function selection module comprises: a feature determination unit, configured to determine a feature of each hash function in the set of hash functions based on the hash function selection network; A hash function selection unit is used to select the at least one hash function from the hash function set based on the difference between the characteristics of the hash functions and the image characteristics of the target image.

8. The device according to claim 6 or 7, characterized in that The search prefix acquisition module includes: a value filling position determining unit, used to determine a value filling position of each hash function in the hash function set in the search prefix; A value filling unit is used to fill a first preset value in the value filling position corresponding to the at least one hash function, and fill a second preset value in the value filling position corresponding to the remaining hash functions in the hash function set except the at least one hash function, to obtain the search prefix.

9. The device according to claim 6 or 7, characterized in that The hash value acquisition module includes: A hash vector acquisition unit, configured to use a twin neural network to perform hash calculation based on the image features of the target image and the at least one hash function to obtain a first hash vector and a second hash vector; a projection unit, configured to project the first hash vector and the second hash vector respectively to obtain a first hash function and a second hash function; A hash value determination unit is used to perform a fitting calculation based on the hash function set, the first hash function and the second hash function to determine the hash value.

10. The device according to claim 6 or 7, characterized in that The hash functions in the hash function set are all locality-sensitive hash functions.

11. An electronic device, characterized in that: include: a memory for storing a computer program product; A processor is used to execute the computer program product stored in the memory, and when the computer program product is executed, it implements the method described in any one of claims 1 to 5.

12. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method described in any one of claims 1 to 5 is implemented.

13. A computer program product, characterized in that The method comprises computer program instructions, which, when executed by a processor, enable the processor to execute the method according to any one of claims 1 to 5.

Images