Image retrieval method and device, equipment and storage medium

Abstract

The embodiment of the invention provides an image retrieval method and device, equipment and a storage medium. The method comprises the steps of determining a first feature of a first image and a second feature of a first similar image in a retrieval image library through a target large model; determining a third feature of a second image and a fourth feature of a second similar image in the to-be-retrieved image library through the target small model; determining a first aggregation mapping feature of the first feature and the second feature, and determining a second aggregation mapping feature of the third feature and the fourth feature; an index table is constructed based on the similarity between the first aggregation mapping feature and the second aggregation mapping feature, a target similar image corresponding to the to-be-retrieved image in the to-be-retrieved image library in the retrieval image library is determined according to the index table, and the similarity between the target similar image and the to-be-retrieved image exceeds a third preset threshold value. The invention aims to improve the retrieval precision and retrieval efficiency of the image.

Description

Technical Field

[0001] The present application relates to the field of financial technology, and in particular to an image retrieval method, an image retrieval device, a computer device, and a computer-readable storage medium. Background Art

[0002] In order to meet the image retrieval needs of customers, financial technology companies, such as insurance companies and banks, usually develop image retrieval systems to serve customers. The image retrieval technology in the image retrieval system is usually symmetric image retrieval, that is, the same model is used to realize the construction of the retrieval library and the feature extraction of the image to be retrieved. However, it is difficult for commonly used mobile devices to support the operation of large-parameter models. Therefore, it is necessary to apply asymmetric retrieval technology, that is, use a large-parameter model to process images on the server side and a small-parameter model to process images on the mobile side, thereby realizing image retrieval.

[0003] However, the commonly used solutions for asymmetric retrieval are mostly based on distillation learning, which is easily limited by the number of model parameters. This may cause the image retrieval results to be affected by the number of samples, resulting in low retrieval accuracy and low retrieval efficiency. Summary of the invention

[0004] The present application provides an image retrieval method, an image retrieval device, a computer device and a computer-readable storage medium, aiming to improve the retrieval accuracy and retrieval efficiency of images.

[0005] To achieve the above purpose, the present application also provides an image retrieval method, comprising:

[0006] Determine a first feature of a first image and a second feature of a first similar image in a search image library by using a target macro model, wherein the first similar image is an image whose similarity to the first image exceeds a first preset threshold;

[0007] Determine, by using the target small model, a third feature of the second image in the to-be-retrieved image library and a fourth feature of the second similar image, wherein the second similar image is an image whose similarity to the second image exceeds a second preset threshold;

[0008] Determine a first aggregate mapping feature of the first feature and the second feature, and determine a second aggregate mapping feature of the third feature and the fourth feature;

[0009] An index table is constructed based on the similarity between the first aggregate mapping feature and the second aggregate mapping feature, and a target similar image corresponding to the image to be retrieved in the image library to be retrieved in the retrieval image library is determined according to the index table, wherein the target similar image is an image whose similarity with the image to be retrieved exceeds a third preset threshold.

[0010] To achieve the above purpose, the present application also provides an image retrieval device, comprising:

[0011] A determination module, configured to determine a first feature of a first image and a second feature of a first similar image in a search image library by using a target macro model, wherein the first similar image is an image whose similarity to the first image exceeds a first preset threshold;

[0012] The determination module is further used to determine the third feature of the second image in the to-be-retrieved image library and the fourth feature of the second similar image through the target small model, wherein the second similar image is an image whose similarity with the second image exceeds a second preset threshold;

[0013] The determining module is further configured to determine a first aggregate mapping feature of the first feature and the second feature, and determine a second aggregate mapping feature of the third feature and the fourth feature;

[0014] A retrieval module is used to construct an index table based on the similarity between the first aggregate mapping feature and the second aggregate mapping feature, and determine the target similar image corresponding to the image to be retrieved in the image library to be retrieved in the retrieval image library according to the index table, wherein the target similar image is an image whose similarity with the image to be retrieved exceeds a third preset threshold.

[0015] In addition, to achieve the above-mentioned purpose, the present application also provides a computer device, which includes a memory and a processor; the memory is used to store a computer program; the processor is used to execute the computer program and implement the steps of any one of the image retrieval methods provided in the embodiments of the present application when executing the computer program.

[0016] In addition, to achieve the above-mentioned purpose, the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the processor implements the steps of the image retrieval method described in any one of the embodiments of the present application.

[0017] The present application discloses an image retrieval method, an image retrieval device, a computer device, and a computer-readable storage medium. The method includes determining a first feature of a first image in a retrieval image library and a second feature of a first similar image through a target large model, wherein the first similar image is an image whose similarity with the first image exceeds a first preset threshold, and at the same time, a third feature of a second image in a to-be-retrieved image library and a fourth feature of a second similar image can be determined through a target small model, wherein the second similar image is an image whose similarity with the second image exceeds a second preset threshold. Further, a first aggregate mapping feature of the first feature and the second feature can be determined, and a second aggregate mapping feature of the third feature and the fourth feature can be determined, and an index table can be constructed based on the similarity of the first aggregate mapping feature and the second aggregate mapping feature, and a target similar image corresponding to the to-be-retrieved image in the to-be-retrieved image library in the retrieval image library can be determined according to the index table, wherein the target similar image is an image whose similarity with the to-be-retrieved image exceeds a third preset threshold. The present application extracts features for the first image and the second image respectively through the combined use of the target large model and the target small model, and can capture the similarity between images more comprehensively and meticulously, thereby improving the accuracy of image retrieval. Furthermore, by aggregating and mapping image features, the computational complexity can be reduced and the retrieval speed can be accelerated, especially when searching in a large-scale image library, which can effectively improve the retrieval efficiency. In addition, by dynamically building an index table based on the similarity of aggregated mapping features, it is possible to quickly locate target similar images to the image to be retrieved, reduce redundant retrieval steps, and further improve the retrieval performance and real-time performance of the overall system. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0019] Figure 1 It is a scene schematic diagram of an image retrieval method provided in an embodiment of the present application;

[0020] Figure 2 It is a flowchart of an image retrieval method provided in an embodiment of the present application;

[0021] Figure 3 is a schematic block diagram of an image retrieval device provided in an embodiment of the present application;

[0022] Figure 4 It is a schematic block diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0023] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0024] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor do they have to be executed in the order described. For example, some operations / steps can also be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions. In addition, although the functional modules are divided in the device schematic, in some cases, the module division can be different from that in the device schematic.

[0025] The term "and / or" as used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0026] In conjunction with the accompanying drawings, some embodiments of the present application are described in detail below. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0027] like Figure 1 As shown, the image retrieval method provided in the embodiment of the present application can be applied to Figure 1In the application environment shown. The application environment includes a terminal device 110 and a server 120, wherein the terminal device 110 can communicate with the server 120 through a network. Specifically, the server 120 can determine the first feature of the first image in the retrieval image library and the second feature of the first similar image through the target large model, wherein the first similar image is an image whose similarity with the first image exceeds a first preset threshold; determine the third feature of the second image in the image library to be retrieved and the fourth feature of the second similar image through the target small model, wherein the second similar image is an image whose similarity with the second image exceeds a second preset threshold; determine the first aggregate mapping feature of the first feature and the second feature, and determine the second aggregate mapping feature of the third feature and the fourth feature; construct an index table based on the similarity of the first aggregate mapping feature and the second aggregate mapping feature, and determine the target similar image corresponding to the image to be retrieved in the image library to be retrieved in the retrieval image library according to the index table, wherein the target similar image is an image whose similarity with the image to be retrieved exceeds a third preset threshold, thereby sending the target similar image to the terminal device 110. Among them, the server 120 can be an independent server, or it can be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms. The terminal device 110 can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, etc., but is not limited to this. The terminal and the server can be directly or indirectly connected via wired or wireless communication, and this application is not limited here.

[0028] See also Figure 2 , Figure 2 FIG. 1 is a flow chart of an image retrieval method provided in an embodiment of the present application. Figure 2 As shown, the image retrieval method can be implemented through steps S11 to S14.

[0029] Step S11: determining a first feature of a first image and a second feature of a first similar image in a retrieved image library through a target macro model.

[0030] The first similar image is an image whose similarity to the first image exceeds a first preset threshold.

[0031] Step S12: Determine the third feature of the second image in the to-be-retrieved image library and the fourth feature of the second similar image by using the target small model.

[0032] The second similar image is an image whose similarity to the second image exceeds a second preset threshold.

[0033] It should be noted that the present application does not limit the first preset threshold and the second preset threshold. For example, the first preset threshold and the second preset threshold may be 80%, 90%, etc. respectively. The present application takes the first preset threshold and the second preset threshold as 80% as an example for illustration. That is to say, the first similar image is an image whose similarity with the first image exceeds 80%, and the second similar image is an image whose similarity with the second image exceeds 80%.

[0034] Furthermore, the target large model is a complex deep learning model that usually has higher computing power and stronger feature extraction capabilities, and is suitable for processing complex image features. Compared with the large model, the target small model usually has a smaller amount of computing and is suitable for processing lighter tasks or scenarios that require efficient response. In this step, the target small model is used to process images in the "image library to be retrieved".

[0035] For example, the first image in the retrieval image library may be processed first, and the first image may be analyzed by the target large model to extract the first feature of the image. The first feature may be a high-dimensional vector related to image vision, such as a multi-dimensional representation of information such as color, texture, shape, and edge. By calculating the similarity between the first image and other images in the retrieval image library, an image whose similarity to the image exceeds a first preset threshold may be determined, namely, a first similar image.

[0036] For example, the target macro model can perform feature extraction on the first similar image to determine the second feature. By comparing the first feature with the second feature, the similarity between the images can be further determined.

[0037] For example, after obtaining the first feature of the first image and the second feature of the first similar image, the large model can also perform comparative analysis on the first feature and the second feature to find out the similarities and differences between the first feature and the second feature, so as to further optimize the retrieval results and improve the accuracy of subsequent operations.

[0038] For example, in the process of image retrieval, the target large model can also use multi-scale feature extraction technology. Different objects or scenes in the image may have different feature expressions at different scales. For example, local features (such as local shape and texture) may be very important in small object recognition, while global features (such as overall color and layout) are more recognizable in large scenes. Therefore, through multi-scale extraction, the model can better capture the details and global information in the image, thereby improving the accuracy of retrieval.

[0039] For example, the target small model processes the second image in the image library to be retrieved, extracts its third feature, realizes feature encoding of the image, and generates a vectorized representation. At the same time, the target small model can also calculate the similarity between the second image and other images in the image library to be retrieved, determine the second similar images, and extract the fourth feature of these similar images and compare them with the previous features.

[0040] The above implementation methods, through the division of labor and cooperation between large models and small models, can more efficiently process image libraries of different sizes and adapt to the efficient retrieval needs in large-scale image libraries.

[0041] Step S13: determining a first aggregate mapping feature of the first feature and the second feature, and determining a second aggregate mapping feature of the third feature and the fourth feature.

[0042] Step S14: constructing an index table based on the similarity between the first aggregate mapping feature and the second aggregate mapping feature, and determining the target similar image in the retrieval image library corresponding to the image to be retrieved in the image to be retrieved library according to the index table.

[0043] The target similar image is an image whose similarity with the image to be retrieved exceeds a third preset threshold.

[0044] It should be noted that the present application does not limit the third preset threshold. For example, the third preset threshold may be 80%, 90%, etc. The present application uses the third preset threshold of 80% as an example for illustration. That is to say, the target similar image is an image whose similarity with the image to be retrieved exceeds 80%.

[0045] Aggregate mapping features are obtained by further processing or combining the extracted features to obtain a more representative feature vector for subsequent similarity comparison.

[0046] For example, by combining or mapping the first feature and the second feature (for example, using weighted average, feature concatenation, or other aggregation methods), a first aggregate mapping feature is generated, which can more comprehensively express the global feature information of the first image and its similar images. Similarly, the third feature and the fourth feature can be combined by the above method to generate a second aggregate mapping feature for expressing the global feature information of the second image and its similar images.

[0047] For example, more features, such as local features, semantic features, or depth features, can be extracted from the image, and a more accurate aggregate feature representation can be obtained through the aggregate mapping of these features. For example, a combination of multiple feature extraction algorithms can be used to weight and fuse different features to generate a comprehensive first aggregate mapping feature or a second aggregate feature, which is not limited in this application.

[0048] For example, by calculating the similarity between the first aggregate mapping features and the second aggregate mapping features (usually using cosine similarity, Euclidean distance, etc.), their degree of similarity can be obtained. This similarity reflects the overall similarity between the two groups of image features (i.e., the first image and its similar images and the second image and its similar images), thereby further helping to determine the correlation between the two.

[0049] For example, based on the calculated similarity values, feature pairs with high similarity can be recorded in an index table, which is used to represent the core data structure used to quickly find similar images in the image retrieval system, and each index item generally includes an image ID in the retrieval image library and an image ID in the image library to be retrieved, as well as a similarity score between them.

[0050] For example, the corresponding target similar image can be found in the retrieval image library through the index table according to the features of the image to be retrieved. By consulting the index table, the image that best meets the similarity requirement can be quickly locked, thereby completing the retrieval task.

[0051] The above implementation method can improve the accuracy of image retrieval through the aggregation mapping of multiple features. In addition, constructing an index table allows the system to complete the matching of similar images through fast table lookup during the retrieval process, reducing the overhead of repeated calculations.

[0052] The image retrieval method disclosed in the embodiment of the present application includes determining the first feature of the first image and the second feature of the first similar image in the retrieval image library through the target large model, wherein the first similar image is an image whose similarity with the first image exceeds the first preset threshold, and at the same time, the third feature of the second image and the fourth feature of the second similar image in the image library to be retrieved can be determined through the target small model, wherein the second similar image is an image whose similarity with the second image exceeds the second preset threshold. Further, the first aggregate mapping feature of the first feature and the second feature can be determined, and the second aggregate mapping feature of the third feature and the fourth feature can be determined, and an index table is constructed based on the similarity of the first aggregate mapping feature and the second aggregate mapping feature, and the target similar image corresponding to the image to be retrieved in the image library to be retrieved in the retrieval image library is determined according to the index table, wherein the target similar image is an image whose similarity with the image to be retrieved exceeds the third preset threshold. The present application extracts features for the first image and the second image respectively through the combined use of the target large model and the target small model, and can capture the similarity between images more comprehensively and meticulously, thereby improving the accuracy of image retrieval. Furthermore, by aggregating and mapping image features, the computational complexity can be reduced and the retrieval speed can be accelerated, especially when searching in a large-scale image library, which can effectively improve the retrieval efficiency. In addition, by dynamically building an index table based on the similarity of aggregated mapping features, it is possible to quickly locate target similar images to the image to be retrieved, reduce redundant retrieval steps, and further improve the retrieval performance and real-time performance of the overall system.

[0053] Optionally, determining a first aggregate mapping feature of the first feature and the second feature, and determining a second aggregate mapping feature of the third feature and the fourth feature, includes: generating the first aggregate mapping feature based on the first feature and the second feature through the aggregation model corresponding to the target large model and the first mapping model, and generating the second aggregate mapping feature based on the third feature and the fourth feature through the second mapping model corresponding to the target small model.

[0054] The aggregation model is a part of the target large model or a sub-model associated with it, which is specifically used to process the aggregation process of multiple groups of features. In the embodiment of the present application, the aggregation model is used to process the first feature and the second feature, that is, the features of the retrieved image and its similar images. The purpose of the aggregation model is to integrate multiple features to form a more complete feature vector that can better reflect the global features of the image.

[0055] It should be noted that the aggregation model may include weighted summation, feature concatenation, or high-order feature fusion using a deep neural network, and this application does not limit this.

[0056] Furthermore, the first mapping model is a model used in the large model to map and transform the aggregated features. Its function is to transform the comprehensive features generated by the aggregation model into a standardized or compressed feature vector, which is usually used for subsequent similarity calculations. The mapping model may include dimensionality reduction technology, nonlinear transformation or other neural network structures, with the purpose of generating a fixed-length feature vector for easy comparison with other features.

[0057] For example, the aggregation model generates an aggregated feature representation based on the first feature and the second feature, and the aggregated feature integrates the feature information of the first image and images similar to it, and can more completely describe the similarity between the images. Then, the first mapping model further maps the aggregated feature to generate a final first aggregated mapping feature, which is used for subsequent similarity calculations.

[0058] For example, the second mapping model generates a new aggregate feature according to the third feature and the fourth feature of the target small model, and then further maps the feature to form a second aggregate mapping feature for similarity calculation with the first aggregate mapping feature.

[0059] In the above implementation, the target large model and the small model process and transform features through the aggregation model and the mapping model respectively, so that the system can handle complex image retrieval tasks more comprehensively and efficiently. The aggregation model can effectively combine multiple features, and the mapping model ensures the comparability and computational efficiency of the features.

[0060] Optionally, constructing an index table based on the similarity between the first aggregate mapping feature and the second aggregate mapping feature includes: determining the similarity between the first aggregate mapping feature and the second aggregate mapping feature based on the cosine distance; in response to the similarity being a maximum value, establishing a correspondence between the first image and the second image; and constructing an index table based on several correspondences.

[0061] In the above method, by constructing an index table, the correspondence between image features can be pre-established. In actual retrieval, the target image and similar images can be quickly matched, reducing the time overhead of real-time similarity calculation, thereby improving retrieval efficiency. In addition, using cosine distance for similarity calculation can more accurately capture the feature similarity of different images, especially when the feature representation is enhanced by aggregating mapping features, the accuracy of the retrieval results is further improved.

[0062] Optionally, the target large model and the target small model are obtained in the following manner: obtaining an initial large model, an initial small model and a training set, wherein the training set includes several images; annotating the training set to obtain corresponding annotation results, wherein the annotation results include similar images corresponding to each image; and training the initial large model and the initial small model respectively using the training set and the annotation results to obtain the target large model and the target small model.

[0063] Specifically, the annotation results corresponding to the training set can be used as labels for the group of input data, and then each group of labeled training sets can be input into the initial large model and the initial small model for supervised learning. When the training end conditions are met, such as the number of training times reaches a threshold or the output accuracy of the model reaches an accuracy threshold, the training is terminated to obtain the trained target large model and target small model.

[0064] In an embodiment of the present application, the training set and the annotation results can be input into the initial large model and the initial small model for supervised learning, and then the target large model and the target small model are trained.

[0065] Optionally, after obtaining the target large model and the target small model, it also includes: iteratively training the target small model and the target large model to extract data features, and calculating a first loss function and a second loss function; iteratively training the first loss function and the second loss function using a preset method for the purpose of reducing the value of the loss function until the expected threshold requirement is met; obtaining the iterative target large model based on the iteratively trained first loss function, and obtaining the iterative target small model based on the iteratively trained second loss function.

[0066] It can be understood that in order to train the target small model and the target large model with higher accuracy, the target small model and the target large model can be repeatedly iterated to train so that the loss function is continuously reduced until the loss function meets the expected threshold requirement.

[0067] It should be noted that the present application does not limit the above-mentioned preset method and expected threshold. For example, the preset method may be a gradient descent algorithm, a batch gradient descent algorithm, a stochastic gradient descent algorithm, etc. The present application takes the gradient descent algorithm as an example for explanation.

[0068] The purpose of the gradient descent algorithm is to find the minimum value of the loss function by iteration, or to converge to the minimum value. In a geometric sense, the gradient descent algorithm means that the gradient decreases fastest in the direction opposite to the vector where the function changes the fastest, so it is easier to find the minimum value of the function. Based on this, in an embodiment of the present application, the target small model and the target large model can be repeatedly iterated and trained by the gradient descent algorithm so that the loss function is continuously reduced, thereby reducing the error of the calculation result.

[0069] In an embodiment of the present application, the loss function is continuously reduced by repeatedly iteratively training the target small model and the target large model using a gradient descent algorithm to obtain an iterative target small model and an iterative target large model, and then a more accurate video content corresponding to the text input can be obtained based on the iterative target small model and the target large model.

[0070] Optionally, the first loss function is obtained by the following formula:

[0071]

[0072] Among them, L CLS is the first loss function; y is the category to which the first image and the first similar image belong; s and m are artificially set hyperparameters. It is the spatial angle between the target large model feature and the target small model feature.

[0073] Optionally, the second loss function is obtained by the following formula:

[0074]

[0075] Among them, L sim is the second loss function; f TQ-I is the third characteristic or the fourth characteristic; f T-MG-IK is the second aggregate mapping feature.

[0076] See also Figure 3 , Figure 3 1 is a schematic block diagram of an image retrieval device provided in an embodiment of the present application. The image retrieval device can be configured in a server to execute the aforementioned image retrieval method.

[0077] like Figure 4 As shown, the image retrieval device 200 includes: a determination module 201 and a retrieval module 202 .

[0078] The determination module 201 is used to determine a first feature of a first image and a second feature of a first similar image in a search image library through a target large model, wherein the first similar image is an image whose similarity with the first image exceeds a first preset threshold; determine a third feature of a second image and a fourth feature of a second similar image in a search image library through a target small model, wherein the second similar image is an image whose similarity with the second image exceeds a second preset threshold; determine a first aggregate mapping feature of the first feature and the second feature, and determine a second aggregate mapping feature of the third feature and the fourth feature;

[0079] The retrieval module 202 is used to construct an index table based on the similarity between the first aggregate mapping feature and the second aggregate mapping feature, and determine the target similar image corresponding to the image to be retrieved in the image library to be retrieved in the retrieval image library according to the index table, wherein the target similar image is an image whose similarity with the image to be retrieved exceeds a third preset threshold.

[0080] The retrieval module 202 is used to generate the first aggregate mapping feature according to the first feature and the second feature through the aggregation model and the first mapping model corresponding to the target large model, and to generate the second aggregate mapping feature according to the third feature and the fourth feature through the second mapping model corresponding to the target small model.

[0081] The retrieval module 202 is used to determine the similarity between the first aggregate mapping feature and the second aggregate mapping feature according to the cosine distance; in response to the similarity being the maximum value, establish a correspondence between the first image and the second image; and construct the index table according to a plurality of the correspondences.

[0082] The determination module 202 is also used to obtain an initial large model, an initial small model and a training set, wherein the training set includes a plurality of images; annotate the training set to obtain corresponding annotation results, wherein the annotation results include similar images corresponding to each of the images; and train the initial large model and the initial small model respectively through the training set and the annotation results to obtain the target large model and the target small model.

[0083] The determination module 202 is also used to iteratively train the target small model and the target large model to extract data features, and calculate a first loss function and a second loss function; iteratively train the first loss function and the second loss function using a preset method for the purpose of reducing the value of the loss function until the expected threshold requirement is met; based on the iteratively trained first loss function, the iterative target large model is obtained, and based on the iteratively trained second loss function, the iterative target small model is obtained.

[0084] It should be noted that those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices and modules and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0085] The method and apparatus of the present application can be used in many general or special computing system environments or configurations, such as personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer terminal devices, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices.

[0086] Exemplarily, the above method and apparatus may be implemented in the form of a computer program. The computer program may be implemented in the form of a computer program. Figure 4 Runs on the computer device shown.

[0087] See also Figure 4 , Figure 4 1 is a schematic diagram of a computer device provided in an embodiment of the present application. The computer device may be a server.

[0088] like Figure 4 As shown, the computer device includes a processor, a memory and a network interface connected through a system bus, wherein the memory may include a volatile storage medium, a non-volatile storage medium and an internal memory.

[0089] The non-volatile storage medium can store an operating system and a computer program. The computer program includes program instructions, and when the program instructions are executed, the processor can execute any image retrieval method.

[0090] The processor is used to provide computing and control capabilities and support the operation of the entire computer equipment.

[0091] The internal memory provides an environment for the operation of the computer program in the non-volatile storage medium. When the computer program is executed by the processor, the processor can execute any image retrieval method.

[0092] The network interface is used for network communication, such as sending assigned tasks, etc. Those skilled in the art can understand that the structure of the computer device is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0093] It should be understood that the processor may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0094] Among them, in some embodiments, the processor is used to determine the first feature of the first image and the second feature of the first similar image in the retrieval image library through the target large model, wherein the first similar image is an image whose similarity with the first image exceeds a first preset threshold; determine the third feature of the second image and the fourth feature of the second similar image in the to-be-retrieved image library through the target small model, wherein the second similar image is an image whose similarity with the second image exceeds a second preset threshold; determine the first aggregate mapping feature of the first feature and the second feature, and determine the second aggregate mapping feature of the third feature and the fourth feature; construct an index table based on the similarity of the first aggregate mapping feature and the second aggregate mapping feature, and determine the target similar image corresponding to the to-be-retrieved image in the to-be-retrieved image library in the retrieval image library according to the index table, wherein the target similar image is an image whose similarity with the to-be-retrieved image exceeds a third preset threshold.

[0095] In some embodiments, the processor is also used to generate the first aggregate mapping feature based on the first feature and the second feature through the aggregation model and the first mapping model corresponding to the target large model, and to generate the second aggregate mapping feature based on the third feature and the fourth feature through the second mapping model corresponding to the target small model.

[0096] In some embodiments, the processor is also used to determine the similarity between the first aggregate mapping feature and the second aggregate mapping feature based on the cosine distance; in response to the similarity being a maximum value, establish a correspondence between the first image and the second image; and construct the index table based on several of the correspondences.

[0097] In some embodiments, the processor is also used to obtain an initial large model, an initial small model and a training set, wherein the training set includes a number of images; annotate the training set to obtain corresponding annotation results, wherein the annotation results include similar images corresponding to each of the images; and train the initial large model and the initial small model respectively through the training set and the annotation results to obtain the target large model and the target small model.

[0098] In some embodiments, the processor is also used to iteratively train the target small model and the target large model to extract data features, and calculate a first loss function and a second loss function; use a preset method to iteratively train the first loss function and the second loss function for the purpose of reducing the loss function value until the expected threshold requirement is met; based on the iteratively trained first loss function, the iterative target large model is obtained, and based on the iteratively trained second loss function, the iterative target small model is obtained.

[0099] An embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. The computer program includes program instructions, and when the program instructions are executed, any image retrieval method provided in the embodiment of the present application is implemented.

[0100] The computer-readable storage medium may be an internal storage unit of the computer device described in the foregoing embodiment, such as a hard disk or memory of the computer device. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a smart memory card (SmartMedia Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc., equipped on the computer device.

[0101] Furthermore, the computer-readable storage medium may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application program required for at least one function, and the like.

[0102] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A method for retrieving an image, characterized in that: The method comprises: Determine a first feature of a first image and a second feature of a first similar image in a search image library by using a target macro model, wherein the first similar image is an image whose similarity to the first image exceeds a first preset threshold; Determine, by using the target small model, a third feature of the second image in the to-be-retrieved image library and a fourth feature of the second similar image, wherein the second similar image is an image whose similarity to the second image exceeds a second preset threshold; Determine a first aggregate mapping feature of the first feature and the second feature, and determine a second aggregate mapping feature of the third feature and the fourth feature; An index table is constructed based on the similarity between the first aggregate mapping feature and the second aggregate mapping feature, and a target similar image corresponding to the image to be retrieved in the image library to be retrieved in the retrieval image library is determined according to the index table, wherein the target similar image is an image whose similarity with the image to be retrieved exceeds a third preset threshold.

2. The method according to claim 1, characterized in that The determining of the first aggregate mapping feature of the first feature and the second feature, and the determining of the second aggregate mapping feature of the third feature and the fourth feature, comprises: generating the first aggregate mapping feature according to the first feature and the second feature through the aggregation model corresponding to the target large model and the first mapping model, and, The second aggregate mapping feature is generated according to the third feature and the fourth feature through the second mapping model corresponding to the target small model.

3. The method according to claim 1, characterized in that The constructing an index table based on the similarity between the first aggregation mapping feature and the second aggregation mapping feature includes: Determining a similarity between the first aggregate mapping feature and the second aggregate mapping feature according to a cosine distance; In response to the similarity being a maximum value, establishing a corresponding relationship between the first image and the second image; The index table is constructed according to a plurality of the corresponding relationships.

4. The method according to claim 1, characterized in that: The target large model and the target small model are obtained in the following manner: Acquire an initial large model, an initial small model, and a training set, wherein the training set includes a plurality of images; Annotating the training set to obtain corresponding annotation results, wherein the annotation results include similar images corresponding to each of the images; The initial large model and the initial small model are trained respectively using the training set and the annotation results to obtain the target large model and the target small model.

5. The method according to claim 1, characterized in that After obtaining the target large model and the target small model, the method further includes: Iteratively training the target small model and the target large model to extract data features, and calculating a first loss function and a second loss function; Iteratively training the first loss function and the second loss function by using a preset method for the purpose of reducing the value of the loss function until the expected threshold requirement is met; Based on the first loss function after iterative training, the target large model after iteration is obtained, and based on the second loss function after iterative training, the target small model after iteration is obtained.

6. The method according to claim 5, characterized in that The first loss function is obtained by the following formula: Among them, L CLS is the first loss function; y is the category to which the first image and the first similar image belong; s and m are artificially set hyperparameters, is the spatial angle between the target large model feature and the target small model feature.

7. The method according to claim 5, characterized in that The second loss function is obtained by the following formula: Among them, L sim is the second loss function; f TQ-I is the third feature or the fourth feature; T-MG-IK Map features for the second aggregation.

8. An image retrieval device, characterized in that: The retrieval device comprises: A determination module, configured to determine a first feature of a first image and a second feature of a first similar image in a search image library by using a target macro model, wherein the first similar image is an image whose similarity to the first image exceeds a first preset threshold; The determination module is further used to determine the third feature of the second image in the to-be-retrieved image library and the fourth feature of the second similar image through the target small model, wherein the second similar image is an image whose similarity with the second image exceeds a second preset threshold; The determining module is further configured to determine a first aggregate mapping feature of the first feature and the second feature, and determine a second aggregate mapping feature of the third feature and the fourth feature; A retrieval module is used to construct an index table based on the similarity between the first aggregate mapping feature and the second aggregate mapping feature, and determine the target similar image corresponding to the image to be retrieved in the image library to be retrieved in the retrieval image library according to the index table, wherein the target similar image is an image whose similarity with the image to be retrieved exceeds a third preset threshold.

9. A computer device, characterized in that: include: A memory and a processor; wherein the memory is connected to the processor and is used to store a program; the processor is used to implement the steps of the image retrieval method as described in any one of claims 1 to 7 by running the program stored in the memory.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor implements the steps of the image retrieval method according to any one of claims 1 to 7.

Images