Image retrieval method, device, electronic device, storage medium and program product

Abstract

The disclosed embodiments disclose an image retrieval method, device, electronic device, storage medium and program product, wherein the method comprises: using a hash function selection network to select at least one hash function corresponding to a target image from a hash function set; generating a retrieval prefix based on the at least one hash function; using a hash calculation network to perform fitting calculation on the image features of the target image based on the hash function set to obtain a hash value; concatenating the retrieval prefix and the hash value to obtain a retrieval index; and performing image retrieval based on the retrieval index. The disclosed embodiments can greatly improve retrieval efficiency while ensuring image retrieval accuracy.

Description

Technical Field

[0001] The present disclosure relates to artificial intelligence technology, and in particular to an image retrieval method, device, electronic device, storage medium and program product. Background Art

[0002] Traditional image retrieval first manually annotates the image with text, and then uses keywords to retrieve the image. This method of providing retrieval results based on the degree of character matching in the image description is very time-consuming and easily affected by the user's subjective influence, resulting in inaccurate text annotations or inaccurate keywords, which in turn leads to low retrieval accuracy.

[0003] Given a query image, it is a very popular research direction to quickly find a certain number of images with similar content from a large number of image databases through image features. In related technologies, an image retrieval method based on Locality-Sensitive Hashing (LSH) is used. When performing image retrieval, all hash functions in the hash function set are hashed, which requires a large amount of data calculation and results in a slow retrieval speed. Summary of the invention

[0004] The embodiments of the present disclosure provide an image retrieval method, device, electronic device, storage medium and program product to solve the above problems.

[0005] According to a first aspect of the present disclosure, there is provided an image retrieval method, comprising:

[0006] Using a hash function selection network, selecting at least one hash function corresponding to the target image from a set of hash functions;

[0007] Based on the at least one hash function, generating a search prefix;

[0008] Using a hash calculation network, performing fitting calculation on the image features of the target image based on the hash function set to obtain a hash value, wherein the hash function selection network and the hash calculation network are jointly trained based on a sample image;

[0009] Concatenate the search prefix and the hash value to obtain a search index;

[0010] Image retrieval is performed based on the retrieval index.

[0011] In some embodiments of the present disclosure, the step of selecting at least one hash function corresponding to the target image from a set of hash functions by using a hash function selection network includes:

[0012] Determining a characteristic of each hash function in the set of hash functions based on the hash function selection network;

[0013] The at least one hash function is selected from the set of hash functions based on a difference between the characteristics of the hash functions and the image characteristics of the target image.

[0014] In some embodiments of the present disclosure, generating a search prefix based on the at least one hash function includes:

[0015] Determine a value filling position of each hash function in the hash function set in the search prefix;

[0016] A first preset value is filled in the value filling position corresponding to the at least one hash function, and a second preset value is filled in the value filling position corresponding to the remaining hash functions in the hash function set except the at least one hash function, to obtain the search prefix.

[0017] In some embodiments of the present disclosure, the using a hash computing network to perform fitting calculation on the image features of the target image based on the hash function set includes:

[0018] Using a twin neural network, performing hash calculation based on the image features of the target image and the hash function set to obtain a first hash vector and a second hash vector;

[0019] Projecting the first hash vector and the second hash vector respectively to obtain a first hash function and a second hash function;

[0020] A fitting calculation is performed based on the hash function set, the first hash function, and the second hash function to determine the hash value.

[0021] In some embodiments of the present disclosure, all hash functions in the hash function set are locality sensitive hash functions.

[0022] According to a second aspect of the embodiments of the present disclosure, there is provided an image retrieval device, comprising:

[0023] A hash function selection module, used to select at least one hash function corresponding to the target image from the hash function set using a hash function selection network;

[0024] A search prefix acquisition module, used to generate a search prefix based on the at least one Hash function;

[0025] A hash value acquisition module, used for performing fitting calculation on the image features of the target image based on the hash function set using a hash calculation network, wherein the hash function selection network and the hash calculation network are obtained by joint training based on sample images;

[0026] A search index acquisition module, used to concatenate the search prefix and the hash value to obtain a search index;

[0027] The image retrieval module is used to perform image retrieval based on the retrieval index.

[0028] In some embodiments of the present disclosure, the hash function selection module includes:

[0029] a feature determination unit, configured to determine a feature of each hash function in the set of hash functions based on the hash function selection network;

[0030] A hash function selection unit is used to select the at least one hash function from the hash function set based on the difference between the characteristics of the hash functions and the image characteristics of the target image.

[0031] In some embodiments of the present disclosure, the search prefix acquisition module includes:

[0032] a value filling position determining unit, used to determine a value filling position of each hash function in the hash function set in the search prefix;

[0033] A value filling unit is used to fill a first preset value in the value filling position corresponding to the at least one hash function, and fill a second preset value in the value filling position corresponding to the remaining hash functions in the hash function set except the at least one hash function, to obtain the search prefix.

[0034] In some embodiments of the present disclosure, the hash value acquisition module includes:

[0035] A hash vector acquisition unit, configured to use a twin neural network to perform a hash calculation based on the image features of the target image and the at least one hash function to obtain a first hash vector and a second hash vector;

[0036] a projection unit, configured to project the first hash vector and the second hash vector respectively to obtain a first hash function and a second hash function;

[0037] A hash value determination unit is used to perform a fitting calculation based on the hash function set, the first hash function and the second hash function to determine the hash value.

[0038] In some embodiments of the present disclosure, all hash functions in the hash function set are locality sensitive hash functions.

[0039] According to a third aspect of the present disclosure, an electronic device is provided, including:

[0040] a memory for storing a computer program product;

[0041] The processor is used to execute the computer program product stored in the memory, and when the computer program product is executed, the method described in the first aspect above is implemented.

[0042] According to a fourth aspect of the embodiments of the present disclosure, a computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the method described in the first aspect is implemented.

[0043] According to a fifth aspect of the embodiments of the present disclosure, a computer program product is provided, comprising computer program instructions, which, when executed by a processor, enable the processor to execute the method described in the first aspect.

[0044] The image retrieval method, device, electronic device, storage medium and program product of the embodiments of the present disclosure, since the retrieval efficiency will be affected if hash calculation is performed on all hash functions in the hash function set when performing image retrieval, some hash functions that match the target image are selected from the hash function set to generate a retrieval prefix, and the hash function set is fitted according to the image features of the target image to obtain a hash value, and then the retrieval prefix and the hash value are combined to obtain a retrieval index. The image retrieval method that effectively utilizes the retrieval prefix can greatly improve the retrieval efficiency while ensuring the image retrieval accuracy.

[0045] The technical solution of the present disclosure is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] The accompanying drawings, which constitute a part of the specification, illustrate embodiments of the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0047] The present disclosure may be more clearly understood from the following detailed description with reference to the accompanying drawings, in which:

[0048] Figure 1 A schematic diagram of a flow chart of an image retrieval method in one embodiment of the present disclosure;

[0049] Figure 2 is a flowchart of step S1 in some embodiments of the present disclosure;

[0050] Figure 3 is a flowchart of step S2 in some embodiments of the present disclosure;

[0051] Figure 4 is a flowchart of step S3 in some embodiments of the present disclosure;

[0052] Figure 5 is a diagram showing the calculation principle of a hash value in an example of the present disclosure;

[0053] Figure 6 is a structural block diagram of an image retrieval device in some embodiments of the present disclosure;

[0054] Figure 7 is a structural block diagram of a hash function selection module 100 in some embodiments of the present disclosure;

[0055] Figure 8 is a structural block diagram of a search prefix acquisition module 200 in some embodiments of the present disclosure;

[0056] Fig. 9 is a structural block diagram of a hash value acquisition module 300 in some embodiments of the present disclosure;

[0057] Fig.10 The structure block diagram of an electronic device in one embodiment of the present disclosure is shown in FIG. DETAILED DESCRIPTION

[0058] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that the relative arrangement of components and steps, numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present disclosure unless otherwise specifically stated.

[0059] Those skilled in the art can understand that the terms "first" and "second" in the embodiments of the present disclosure are only used to distinguish different steps, devices or modules, etc., and neither represent any specific technical meaning nor indicate the necessary logical order between them.

[0060] It should also be understood that in the embodiments of the present disclosure, “plurality” may refer to two or more than two, and “at least one” may refer to one, two, or more than two.

[0061] It should also be understood that any component, data or structure mentioned in the embodiments of the present disclosure can generally be understood as one or more, unless explicitly limited or otherwise indicated in the context.

[0062] In addition, the term "and / or" in the present disclosure is only a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in the present disclosure generally indicates that the associated objects before and after are in an "or" relationship.

[0063] It should also be understood that the description of the various embodiments in the present disclosure focuses on the differences between the various embodiments, and the same or similar aspects thereof can be referenced to each other, and for the sake of brevity, they will not be described one by one.

[0064] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the present disclosure, its application, or uses.

[0065] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered part of the specification.

[0066] It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0067] The embodiments of the present disclosure can be applied to electronic devices such as terminal devices, computer systems, servers, etc., which can operate with many other general or special computing system environments or configurations. Examples of well-known terminal devices, computing systems, environments and / or configurations suitable for use with electronic devices such as terminal devices, computer systems, servers, etc. include, but are not limited to: personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, microprocessor-based systems, set-top boxes, programmable consumer electronics, network personal computers, small computer systems, large computer systems, and distributed cloud computing technology environments including any of the above systems, etc.

[0068] Electronic devices such as terminal devices, computer systems, servers, etc. can be described in the general context of computer system executable instructions (such as program modules) executed by computer systems. Generally, program modules can include routines, programs, object programs, components, logic, data structures, etc., which perform specific tasks or implement specific abstract data types. Computer systems / servers can be implemented in distributed cloud computing environments, where tasks are performed by remote processing devices linked through a communication network. In a distributed cloud computing environment, program modules can be located on local or remote computing system storage media including storage devices.

[0069] Figure 1 FIG. 1 is a flow chart of an image retrieval method in one embodiment of the present disclosure. Figure 1 As shown, the image retrieval method comprises the following steps:

[0070] S1: Using a hash function selection network, select at least one hash function corresponding to a target image from a hash function set.

[0071] Set a hash function set for image retrieval. The hash function set includes multiple hash functions, and each hash function has different characteristics when applied to image retrieval.

[0072] Input the target image into the pre-trained hash function selection network. Using the hash function selection network, according to the function selection conditions corresponding to the hash function selection network, select at least one hash function corresponding to the target image from the set of hash functions.

[0073] S2: Generate a retrieval prefix based on at least one hash function.

[0074] Sort all the hash functions in the set of hash functions and assign each hash function a corresponding identifier. Generate a retrieval prefix with the sorting relationship of the hash functions according to the positions of the at least one selected hash function in the set of hash functions.

[0075] In an example of the present disclosure, the set of hash functions includes n hash functions, and the identifiers of the n functions are respectively h 1, h 2, …, h n, where n is an integer greater than 2. The at least one hash function selected from the set of hash functions includes h i, h j and h k, where 1 ≤ i < j < k ≤ n. A retrieval prefix such as Z, …, h i, …, h j, …, h k, …, Z can be generated according to the positions of the functions h i, h j, h k in the set of hash functions. Here, Z is a preset value indicating that the function is not selected.

[0076] S3: Use the hash calculation network to perform fitting calculations on the image features of the target image based on the set of hash functions to obtain a hash value. Among them, the hash function selection network and the hash calculation network are jointly trained based on sample images.

[0077] After extracting the image features of the target image using the hash calculation network, according to the function fitting calculation method corresponding to the hash calculation network, perform fitting on the set of hash functions. Then, perform hash calculation on the image features of the target image to obtain a hash value.

[0078] The hash function selection network and the hash calculation network are obtained by joint training based on the sample image. The joint training process of the hash function selection network and the hash calculation network may include: using the initial hash function selection network, selecting at least one hash function corresponding to the sample image from the hash function set, and generating a search prefix according to the selected at least one hash function; using the initial hash calculation network, fitting and calculating the image features of the sample image based on the hash function set to obtain a hash value, concatenating the search prefix and the hash value and searching in the image library, adjusting the parameters of the initial hash function selection network and the initial hash calculation network according to the search results, and obtaining the hash function selection network and the hash calculation network.

[0079] S4: Concatenate the search prefix and the hash value to obtain a search index.

[0080] S5: Perform image retrieval based on the retrieval index.

[0081] According to the search index, a search is performed on the network or in a specified image library to obtain multiple images with features similar to the target image.

[0082] In this embodiment, since the retrieval efficiency will be affected if hash calculation is performed on all hash functions in the hash function set during image retrieval, some hash functions that match the target image are selected from the hash function set to generate a retrieval prefix, and the hash function set is fitted according to the image features of the target image to obtain a hash value, and then the retrieval prefix and the hash value are combined to obtain a retrieval index. The image retrieval method that effectively utilizes the retrieval prefix can greatly improve the retrieval efficiency while ensuring the image retrieval accuracy.

[0083] Figure 2 is a flow chart of step S1 in some embodiments of the present disclosure. Figure 2 As shown, step S1 includes the following steps:

[0084] S1-1: Determine the characteristics of each hash function in the hash function set based on the hash function selection network.

[0085] The function feature extraction network in the hash function selection network is used to extract the features of each hash function in the hash function set.

[0086] S1-2: Select at least one hash function from a set of hash functions based on differences between features of each hash function and image features of the target image.

[0087] The difference between the characteristics of each hash function and the image characteristics of the target image is determined by using a comparison method used in the hash function selection network to determine the difference between the characteristics of each hash function and the image characteristics of the target image.

[0088] According to the differences between the features of each hash function and the image features of the target image, a preset number of hash functions are selected from the hash function set in descending order of the differences to obtain at least one hash function.

[0089] In this embodiment, when the difference between the characteristics of the hash function and the image characteristics of the target image is large, searching the target image can obtain a better retrieval effect. Therefore, using a hash function selection network to select at least one hash function with a large difference in image characteristics from the target image from a set of hash functions helps to improve retrieval accuracy.

[0090] Figure 3 FIG. 1 is a flow chart of step S2 in some embodiments of the present disclosure. Figure 3 As shown, step S2 includes the following steps:

[0091] S2-1: Determine the value filling position of each hash function in the hash function set in the search prefix.

[0092] A blank search prefix of a length corresponding to the number of hash functions in the hash function set can be generated, and each blank position in the blank search prefix is ​​filled with a value corresponding to whether a hash function in the hash function set is selected.

[0093] S2-2: Fill a first preset value in a value filling position corresponding to at least one hash function, and fill a second preset value in a value filling position corresponding to the remaining hash functions in the hash function set except for at least one hash function, to obtain a search prefix.

[0094] At least one value filling position corresponding to a hash function in a blank search prefix is ​​filled with a first preset value selected by the representation function, and the remaining positions of the blank search prefix are filled with a second preset value that violates the selection of the function.

[0095] In some examples of the present disclosure, the hash function set includes 10 hash functions, and at least one hash function selected by the hash function includes the 3rd, 5th, and 8th hash functions in the hash function set, then the search prefix may be: {0,0,1,0,1,0,0,1,0,0}. Among them, the 3rd, 5th, and 8th hash functions in the hash function set are selected, and the value 1 is filled in the 3rd, 5th, and 8th value filling positions of the hash prefix, and the hash functions corresponding to the remaining positions are not selected, and the value 0 is filled.

[0096] In this embodiment, since the use of hash prefixes can improve retrieval efficiency, a hash prefix consisting of a numerical queue is generated to represent whether the corresponding hash function is selected based on whether each hash function in the hash function set is selected. This can reduce the data storage capacity of the retrieval prefix while improving retrieval efficiency.

[0097] Figure 4 FIG. 1 is a flow chart of step S3 in some embodiments of the present disclosure. Figure 4 As shown, step S3 includes the following steps:

[0098] S3-1: Using the twin neural network, a hash calculation is performed based on the image features of the target image and a set of hash functions to obtain a first hash vector and a second hash vector.

[0099] Figure 5 This is a diagram showing the calculation principle of a hash value in an example of the present disclosure. Figure 5 As shown, the stack trace encoders (stack trace encoder 1 and stack trace encoder 2) corresponding to the stack pair (stack 1 and stack 2) of the twin neural network are encoded to generate the corresponding feature map (feature Figure 1 and Features Figure 2 ), and then provide the feature map to the corresponding multi-network layer (multi-network layer 1 and multi-network layer 2). In addition, the hash selection network provides a set of hash functions to the multi-network layer of the twin neural network, so that the multi-network layer calculates the first hash vector and the second hash vector according to the set of hash functions and the feature map. The first hash vector and the second hash vector are multiple connected binary hash codes, each of which is encoded as b bits in the Hamming space, and b is an integer greater than 1.

[0100] S3-2: Project the first hash vector and the second hash vector respectively to obtain a first hash function and a second hash function.

[0101] The first hash vector and the second hash vector are projected respectively by using a preset projection function to obtain a first hash function and a second hash function.

[0102] S3-3: Perform fitting calculation based on the hash function set, the first hash function, and the second hash function to determine a hash value.

[0103] Since hash vectors are in the Hamming space, i.e., hash vectors are restricted to binary values, {0,1} or {-1,1}, computing the collision probability between two random projection hash functions of the same order h is equivalent to computing the Hamming similarity between two hash vectors H(si) and H(sj), where H(si) and H(sj) represent the first hash vector and the second hash vector.

[0104] Considering that the hash function is b-bit encoding (b>1), that is, it is not limited to a single projection, but b coordinates, it is necessary to generalize the property of b>1. Define the generalized Hamming distance between two hash vectors H(si) and H(sj) as the number of k-order different projection hash functions, perform fitting calculations based on the hash function set, the first hash function, and the second hash function, and obtain the hash value based on the fitting calculation process.

[0105] In this embodiment, since the twin neural network is more robust to the problem of category imbalance and can learn from semantic similarity to estimate the distance between two inputs, the twin neural network is used to perform hash calculation and projection based on image features and a set of hash functions to obtain two hash functions. The distance between the two hash functions can represent the similarity between the two hash functions, and then the hash value that can reasonably represent the image features is obtained by hash function fitting, which helps to improve the image retrieval accuracy.

[0106] In some embodiments of the present disclosure, all hash functions in the hash function set are locality sensitive hash functions.

[0107] In this embodiment, since the local sensitive hash function has the characteristic that if two objects are similar in the original data space, they will also have a high degree of similarity after being converted by the hash function respectively, the use of the local sensitive hash function for image retrieval can greatly improve the image retrieval accuracy.

[0108] Figure 6 is a structural block diagram of an image retrieval device in some embodiments of the present disclosure. Figure 6 As shown, the image retrieval device comprises:

[0109] A hash function selection module 100, configured to select at least one hash function corresponding to a target image from a set of hash functions using a hash function selection network;

[0110] A search prefix acquisition module 200, used to generate a search prefix based on at least one hash function;

[0111] A hash value acquisition module 300 is used to use a hash calculation network to perform fitting calculation on the image features of the target image based on a hash function set, wherein the hash function selection network and the hash calculation network are jointly trained based on sample images;

[0112] A search index acquisition module 400 is used to concatenate the search prefix and the hash value to obtain a search index;

[0113] The image retrieval module 500 is used to perform image retrieval based on a retrieval index.

[0114] Figure 7is a structural block diagram of the hash function selection module 100 in some embodiments of the present disclosure. Figure 7 As shown, the hash function selection module 100 includes:

[0115] A feature determination unit 110, configured to determine a feature of each hash function in the set of hash functions based on a hash function selection network;

[0116] The hash function selection unit 120 is configured to select at least one hash function from the hash function set based on the difference between the characteristics of each hash function and the image characteristics of the target image.

[0117] Figure 8 FIG. 2 is a structural block diagram of the search prefix acquisition module 200 in some embodiments of the present disclosure. Figure 8 As shown, the search prefix acquisition module 200 includes:

[0118] A value filling position determining unit 210, used to determine a value filling position of each hash function in the hash function set in the search prefix;

[0119] The value filling unit 220 is used to fill the first preset value in the value filling position corresponding to at least one hash function, and fill the second preset value in the value filling position corresponding to the remaining hash functions in the hash function set except at least one hash function, to obtain a search prefix.

[0120] Fig. 9 is a structural block diagram of the hash value acquisition module 300 in some embodiments of the present disclosure. Fig. 9 As shown, the hash value acquisition module 300 includes:

[0121] A hash vector acquisition unit 310 is used to perform hash calculation based on image features of the target image and at least one hash function using a twin neural network to obtain a first hash vector and a second hash vector;

[0122] A projection unit 320, configured to project the first hash vector and the second hash vector respectively to obtain a first hash function and a second hash function;

[0123] The hash value determination unit 330 is configured to perform fitting calculation based on the hash function set, the first hash function, and the second hash function to determine the hash value.

[0124] In some embodiments of the present disclosure, all hash functions in the hash function set are locality sensitive hash functions.

[0125] It should be noted that the specific implementation of the image retrieval device of the embodiment of the present disclosure is similar to the specific implementation of the image retrieval method of the embodiment of the present disclosure, and the technical effect of the image retrieval device of the embodiment of the present disclosure is similar to the technical effect of the image retrieval method of the embodiment of the present disclosure. Please refer to the description of the image retrieval method part for details. In order to reduce redundancy, it will not be repeated.

[0126] In addition, an embodiment of the present disclosure further provides an electronic device, including:

[0127] Memory for storing computer programs;

[0128] The processor is used to execute the computer program stored in the memory, and when the computer program is executed, the image retrieval method described in any of the above embodiments of the present disclosure is implemented.

[0129] Below, reference Fig.10 To describe the electronic device according to the embodiment of the present disclosure. Fig.10 As shown, the electronic device includes one or more processors and memory.

[0130] The processor may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device to perform desired functions.

[0131] The memory may store one or more computer program products, and the memory may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, a random access memory (RAM) and / or a cache memory (cache), etc. The non-volatile memory may include, for example, a read-only memory (ROM), a hard disk, a flash memory, etc. One or more computer program products may be stored on the computer-readable storage medium, and the processor may run the computer program product to implement the image retrieval method of each embodiment of the present disclosure described above and / or other desired functions.

[0132] In one example, the electronic device may further include: an input device and an output device, and these components are interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0133] In addition, the input device may also include, for example, a keyboard, a mouse, and the like.

[0134] The output device can output various information to the outside, including determined distance information, direction information, etc. The output device can include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto, and the like.

[0135] Of course, to simplify, Fig.10 Only some of the components related to the present disclosure in the electronic device are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, according to specific application situations, the electronic device may further include any other appropriate components.

[0136] In addition to the above-mentioned methods and devices, an embodiment of the present disclosure may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the image retrieval method according to various embodiments of the present disclosure described in the above part of this specification.

[0137] The computer program product may be written in any combination of one or more programming languages ​​to write program code for performing the operations of the disclosed embodiments, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0138] In addition, an embodiment of the present disclosure may also be a computer-readable storage medium having computer program instructions stored thereon, which, when executed by a processor, enables the processor to execute the steps of the image retrieval method according to various embodiments of the present disclosure described in the above part of this specification.

[0139] The computer readable storage medium can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can include, for example, but is not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0140] The basic principles of the present disclosure are described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, effects, etc. mentioned in the present disclosure are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. are required by each embodiment of the present disclosure. In addition, the specific details disclosed above are only for the purpose of illustration and ease of understanding, and are not limitations. The above details do not limit the present disclosure to the necessity of adopting the above specific details to be implemented.

[0141] Each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the system embodiment, since it basically corresponds to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0142] The block diagrams of the devices, apparatuses, equipment, and systems involved in this disclosure are only illustrative examples and are not intended to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagrams. As will be appreciated by those skilled in the art, these devices, apparatuses, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including," "comprising," "having," and the like are open words, referring to "including but not limited to," and can be used interchangeably therewith. The words "or" and "and" used herein refer to the words "and / or," and can be used interchangeably therewith, unless the context clearly indicates otherwise. The word "such as" used herein refers to the phrase "such as but not limited to," and can be used interchangeably therewith.

[0143] The method and apparatus of the present disclosure may be implemented in many ways. For example, the method and apparatus of the present disclosure may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above order of steps for the method is for illustration only, and the steps of the method of the present disclosure are not limited to the order specifically described above, unless otherwise specifically stated. In addition, in some embodiments, the present disclosure may also be implemented as a program recorded in a recording medium, which includes machine-readable instructions for implementing the method according to the present disclosure. Therefore, the present disclosure also covers a recording medium storing a program for executing the method according to the present disclosure.

[0144] It should also be noted that in the apparatus, device and method of the present disclosure, each component or each step can be decomposed and / or recombined. Such decomposition and / or recombination should be regarded as equivalent solutions of the present disclosure.

[0145] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present disclosure. Therefore, the present disclosure is not intended to be limited to the aspects shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

[0146] The above description has been given for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present disclosure to the forms disclosed herein. Although multiple example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.

Claims

1. An image retrieval method, characterized in that: include: Using a hash function selection network, selecting at least one hash function corresponding to the target image from a set of hash functions; Generating a search prefix based on the at least one hash function includes: determining a value filling position of each hash function in the hash function set in the search prefix; filling a first preset value in the value filling position corresponding to the at least one hash function, and filling a second preset value in the value filling position corresponding to the remaining hash functions in the hash function set except the at least one hash function, to obtain the search prefix; Using a hash calculation network, performing fitting calculation on the image features of the target image based on the hash function set to obtain a hash value, including: using a twin neural network, performing hash calculation based on the image features of the target image and the hash function set to obtain a first hash vector and a second hash vector; projecting the first hash vector and the second hash vector respectively to obtain a first hash function and a second hash function; performing fitting calculation based on the hash function set, the first hash function and the second hash function to determine the hash value; wherein the hash function selection network and the hash calculation network are obtained by joint training based on a sample image; Concatenate the search prefix and the hash value to obtain a search index; Image retrieval is performed based on the retrieval index.

2. The method according to claim 1, characterized in that The method of selecting at least one hash function corresponding to the target image from a set of hash functions by using a hash function selection network includes: Determining a characteristic of each hash function in the set of hash functions based on the hash function selection network; The at least one hash function is selected from the set of hash functions based on a difference between the characteristics of the hash functions and the image characteristics of the target image.

3. The method according to claim 1 or 2, characterized in that: The hash functions in the hash function set are all locality-sensitive hash functions.

4. An image retrieval device, characterized in that: include: A hash function selection module, used to select at least one hash function corresponding to the target image from the hash function set using a hash function selection network; A search prefix acquisition module, used to generate a search prefix based on the at least one Hash function; A hash value acquisition module, used for performing fitting calculation on the image features of the target image based on the hash function set using a hash calculation network, wherein the hash function selection network and the hash calculation network are obtained by joint training based on sample images; A search index acquisition module, used to concatenate the search prefix and the hash value to obtain a search index; An image retrieval module, used for performing image retrieval based on the retrieval index; Wherein, the search prefix acquisition module includes: a value filling position determining unit, used to determine a value filling position of each hash function in the hash function set in the search prefix; a value filling unit, configured to fill a first preset value in a value filling position corresponding to the at least one hash function, and fill a second preset value in a value filling position corresponding to the remaining hash functions in the hash function set except the at least one hash function, to obtain the search prefix; The hash value acquisition module includes: A hash vector acquisition unit, configured to use a twin neural network to perform hash calculation based on the image features of the target image and the at least one hash function to obtain a first hash vector and a second hash vector; a projection unit, configured to project the first hash vector and the second hash vector respectively to obtain a first hash function and a second hash function; A hash value determination unit is used to perform a fitting calculation based on the hash function set, the first hash function and the second hash function to determine the hash value.

5. The device according to claim 4, characterized in that The hash function selection module comprises: a feature determination unit, configured to determine a feature of each hash function in the set of hash functions based on the hash function selection network; A hash function selection unit is used to select the at least one hash function from the hash function set based on the difference between the characteristics of the hash functions and the image characteristics of the target image.

6. The device according to claim 4 or 5, characterized in that The hash functions in the hash function set are all locality-sensitive hash functions.

7. An electronic device, characterized in that: include: a memory for storing a computer program product; A processor is used to execute the computer program product stored in the memory, and when the computer program product is executed, it implements the method described in any one of claims 1 to 3 above.

8. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method described in any one of claims 1 to 3 is implemented.

9. A computer program product, characterized in that The method comprises computer program instructions, which, when executed by a processor, enable the processor to execute the method according to any one of claims 1 to 3.

Images