An image retrieval method, related methods and related devices

Abstract

The present invention provides an image retrieval method, related methods and related devices. Among them, the image retrieval method includes: obtaining an image to be retrieved as a target image; determining the scene to which the target image belongs from several scenes as a target scene; obtaining a feature vector corresponding to the target image based on the feature acquisition method corresponding to the target scene among the feature acquisition methods respectively corresponding to several scenes; determining the similarity between each image in the retrieval image library and the target image based on the feature vector corresponding to the target image and the feature vectors respectively corresponding to each image in the retrieval image library, wherein the feature vector corresponding to an image in the retrieval image library is obtained based on the feature acquisition method corresponding to the scene to which the image belongs; and determining an image retrieval result corresponding to the target image according to the determined similarity. The image retrieval method provided by the present invention has a wide application range and high retrieval accuracy, and the user experience is good.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular, to an image retrieval method, a related method, and related devices. Background Art

[0002] Image retrieval refers to retrieving images with similar characteristics to the image to be retrieved from a retrieval image library.

[0003] The current image retrieval solution is as follows: Feature vectors are respectively extracted from each image in the retrieval image library based on a general feature extraction method to obtain the feature vectors respectively corresponding to each image in the retrieval image library. During retrieval, a feature vector is extracted from the image to be retrieved based on the general feature extraction method, and then, based on the extracted feature vector and the feature vectors respectively corresponding to each image in the retrieval image library, the similarity between the image to be retrieved and each image in the retrieval image library is determined. Finally, an image retrieval result corresponding to the image to be retrieved is determined according to the determined similarity.

[0004] Although the above image retrieval solution has a wide application range, its retrieval accuracy is not high and the user experience is not good. Summary of the Invention

[0005] In view of this, the present invention provides an image retrieval method, a related method, and related devices to solve the problem that although the existing image retrieval solution has a wide application range, its retrieval accuracy is not high. The technical solution is as follows:

[0006] An image retrieval method includes:

[0007] Obtain an image to be retrieved as a target image;

[0008] Determine the scene to which the target image belongs from several scenes as a target scene;

[0009] Based on the feature acquisition method corresponding to the target scene among the feature acquisition methods respectively corresponding to the several scenes, obtain the feature vector corresponding to the target image;

[0010] Based on the feature vector corresponding to the target image and the feature vectors respectively corresponding to each image in the retrieval image library, determine the similarity between each image in the retrieval image library and the target image; wherein, the feature vector corresponding to an image in the retrieval image library is obtained based on the feature acquisition method corresponding to the scene to which the image belongs;

[0011] Determine an image retrieval result corresponding to the target image according to the determined similarity.

[0012] Optionally, in the feature acquisition methods corresponding to the respective scenarios, the feature acquisition method corresponding to the target scenario for acquiring the feature vector corresponding to the target image includes:

[0013] Based on the feature extraction models corresponding to the respective scenarios, the feature extraction model corresponding to the target scenario extracts the feature vector from the target image; wherein, the feature extraction model is trained using the training images of the corresponding scenario.

[0014] For the extracted feature vector, the eigenvalue at the processing position corresponding to the target scenario is attenuated, and the attenuated feature vector is used as the feature vector corresponding to the target image; wherein, the dimensions of the feature vectors extracted based on the feature extraction models corresponding to the respective scenarios are the same, and the processing positions corresponding to different scenarios are different.

[0015] Optionally, the processing position corresponding to the target scenario is determined according to the target scenario, the total number of scenarios, and the dimension of the feature vector extracted from the target image based on the feature extraction model corresponding to the target scenario.

[0016] Optionally, the attenuation processing of the eigenvalue at the processing position corresponding to the target scenario in the extracted feature vector includes:

[0017] Based on a preset weight, the eigenvalue at the processing position corresponding to the target scenario in the extracted feature vector is weighted, wherein the weight is greater than 0 and less than 1.

[0018] Optionally, determining the scenario to which the target image belongs from several scenarios includes:

[0019] Based on a pre-trained scenario classification model, determining the scenario to which the target image belongs; wherein, the scenario classification model is trained using the training images labeled with scenario categories of several scenarios.

[0020] An image similarity determination method includes:

[0021] Obtain a first image;

[0022] Determine the scenario to which the first image belongs from several scenarios as the target scenario;

[0023] Based on the feature acquisition method corresponding to the target scenario in the feature acquisition methods corresponding to the respective scenarios, obtain the feature vector corresponding to the first image;

[0024] Determine the similarity between the first image and the second image based on the feature vector corresponding to the first image and the image feature vector corresponding to the second image; wherein, the image feature vector corresponding to the second image is obtained based on the feature acquisition method corresponding to the scene to which the second image belongs.

[0025] Optionally, in the feature acquisition methods corresponding to the respective scenarios, the feature acquisition method corresponding to the target scenario for acquiring the feature vector corresponding to the first image includes:

[0026] Based on the feature extraction models corresponding to the respective scenarios, the feature extraction model corresponding to the target scenario extracts a feature vector from the first image; wherein, the feature extraction model is trained using the training images of the corresponding scenario.

[0027] For the extracted feature vector, perform an attenuation process on the feature value at the processing position corresponding to the target scenario, and the feature vector after the attenuation process is used as the feature vector corresponding to the first image; wherein, the dimensions of the feature vectors extracted based on the feature extraction models corresponding to the respective scenarios are the same, and the processing positions corresponding to different scenarios are different.

[0028] An image retrieval device includes: an image acquisition module, a scene determination module, a feature acquisition module, a similarity determination module, and a retrieval result determination module;

[0029] The image acquisition module is configured to acquire an image to be retrieved as a target image;

[0030] The scene determination module is configured to determine the scene to which the target image belongs from a plurality of scenes as a target scene;

[0031] The feature acquisition module is configured to acquire the feature vector corresponding to the target image based on the feature acquisition method corresponding to the target scenario in the feature acquisition methods corresponding to the respective scenarios;

[0032] The similarity determination module is configured to determine the similarity between each image in the retrieval image library and the target image based on the feature vector corresponding to the target image and the feature vectors corresponding to each image in the retrieval image library; wherein, the feature vector corresponding to an image in the retrieval image library is obtained based on the feature acquisition method corresponding to the scene to which the image belongs.

[0033] The retrieval result determination module is configured to determine the image retrieval result corresponding to the target image according to the similarity between each image in the retrieval image library and the target image.

[0034] An image retrieval device includes: a memory and a processor;

[0035] The memory is used to store programs;

[0036] The processor is used to execute the program to implement each step of the image retrieval method described in any one of the above.

[0037] A readable storage medium stores a computer program, and when the computer program is executed by a processor, each step of the image retrieval method described in any one of the above is implemented.

[0038] After obtaining the image to be retrieved, i.e., the target image, the image retrieval method provided by the present invention first determines the scene to which the target image belongs as the target scene from several scenes, then obtains the feature acquisition method corresponding to the target scene from the feature acquisition methods corresponding to several scenes respectively, and acquires the feature vector corresponding to the target image. Then, based on the feature vector corresponding to the target image and the feature vectors corresponding to each image in the retrieval image library, the similarity between each image in the retrieval image library and the target image is determined. Finally, according to the determined similarity, the image retrieval result corresponding to the target image is determined. The image retrieval method provided by the present invention is applicable to multiple scenes. Since the image retrieval method provided by the present invention uses the feature acquisition method corresponding to the scene to which the image belongs to acquire the feature vector corresponding to the image, the acquired feature vector can finely represent the features of the object in the image. Based on such a feature vector, fine-grained image retrieval can be realized, and finally a relatively accurate image retrieval result can be obtained. That is, the image retrieval method provided by the present invention has a wide application range and high retrieval accuracy, and the user experience is good. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only the embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained according to the provided drawings without creative efforts.

[0040] Figure 1 It is a schematic diagram of the hardware architecture related to the present invention;

[0041] Figure 2 It is a schematic flowchart of the image retrieval method provided by the embodiment of the present invention;

[0042] Figure 3 It is a schematic flowchart of a specific example of the image retrieval method provided by the embodiment of the present invention;

[0043] Figure 4 It is a schematic flowchart of another specific example of the image retrieval method provided by the embodiment of the present invention;

[0044] Figure 5 Schematic flowchart of the image similarity determination method provided by an embodiment of the present invention;

[0045] Figure 6 Schematic structural diagram of the image retrieval device provided by an embodiment of the present invention;

[0046] Figure 7 Schematic structural diagram of the image retrieval device provided by an embodiment of the present invention. Specific implementation manners

[0047] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0048] The applicant's research on the current image retrieval solution finds that in order to obtain a wider application range, the current image retrieval solution uses a general feature acquisition method to obtain the feature vector corresponding to the image. Among them, the general feature extraction method is generally as follows: training a feature extraction model using the training images of multiple scenarios, using the trained feature extraction model to extract the feature vector of the image, and the extracted feature vector is used as the feature vector corresponding to the image. However, training a feature extraction model using the training images of multiple scenarios can only extract a relatively rough feature vector that characterizes the object in the image, and cannot extract a relatively fine feature vector that characterizes the object in the image. This results in that the current image retrieval solution cannot achieve refined retrieval, that is, the retrieval accuracy is not high. For example, when the image to be retrieved is the face image of A, the retrieval results obtained by using the current image retrieval solution are basically all face images, but it is very likely not the face image of user A. Another example is that when the image to be retrieved is a vehicle image, the retrieval results obtained by using the current image retrieval solution are basically all vehicle images, but it is very likely not the vehicle image with the same angle, color, brand, etc. as the vehicle in the image to be retrieved.

[0049] In view of the low retrieval accuracy of the current image retrieval solution, the applicant attempts to propose an image retrieval method with a wider application range (that is, applicable to multiple scenarios) and a higher retrieval accuracy. For this reason, the applicant conducts research and finally proposes an image retrieval method with good effects. This image retrieval method can achieve multi-scenario image retrieval and has a high retrieval accuracy. Before introducing the image retrieval method provided by the present invention, the hardware architecture involved in the present invention will be described first.

[0050] In one possible implementation manner, as Figure 1As shown in the figure, the hardware architecture involved in the present invention may include: an electronic device 101 and a server 102.

[0051] Exemplarily, the electronic device 101 may be any kind of electronic product that can perform human-computer interaction with a user in one or more ways such as a keyboard, a touchpad, a touch screen, a remote control, voice interaction, or a handwriting device. For example, a mobile phone, a laptop computer, a tablet computer, a handheld computer, a personal computer, a wearable device, a smart TV, a PAD, etc.

[0052] It should be noted that Figure 1 merely as an example, there can be various types of electronic devices, not limited to Figure 1 the tablet computer in

[0053] Exemplarily, the server 102 may be a single server, or a server cluster composed of multiple servers, or a cloud computing server center. The server 102 may include a processor, a memory, and a network interface, etc.

[0054] Exemplarily, the electronic device 101 may establish a connection and communicate with the server 102 through a wireless communication network; exemplarily, the electronic device 101 may establish a connection and communicate with the server 102 through a wired communication network.

[0055] The electronic device 101 can obtain the image to be retrieved, send the image to be retrieved to the server 102, and the server 102 performs image retrieval according to the image retrieval method provided by the present invention, and sends the image retrieval result to the electronic device 101.

[0056] In another possible implementation manner, the hardware architecture involved in the present invention may include: an electronic device. The electronic device is a device with strong data processing capabilities.

[0057] Exemplarily, the electronic device may be any kind of electronic product that can perform human-computer interaction with a user in one or more ways such as a keyboard, a touchpad, a touch screen, a remote control, voice interaction, or a handwriting device. For example, a mobile phone, a laptop computer, a tablet computer, a handheld computer, a personal computer, a wearable device, a smart TV, a PAD, etc.

[0058] The electronic device obtains the image to be retrieved, and performs image retrieval according to the image retrieval method provided by the present invention to obtain an image retrieval result.

[0059] Those skilled in the art should understand that the above-mentioned electronic devices and servers are only examples, and other existing or future possible electronic devices or servers that can be applied to the present invention should also be included within the protection scope of the present invention, and are hereby incorporated herein by reference.

[0060] The image retrieval method provided by the present invention will be introduced through the following embodiments.

[0061] First Embodiment

[0062] Please refer to Figure 2 , which shows a schematic flowchart of the image retrieval method provided by an embodiment of the present invention. The method may include:

[0063] Step S201: Obtain the image to be retrieved as the target image.

[0064] The image to be retrieved may be an image of any one of several scenarios.

[0065] Exemplarily, the several scenarios may include person re-identification scenarios, vehicle re-identification scenarios, face recognition scenarios, product retrieval scenarios, landmark retrieval scenarios, clothing retrieval scenarios, etc. The image to be retrieved may be an image of any scenario. For example, a face image in a face recognition scenario.

[0066] It should be noted that the above person re-identification scenarios, vehicle re-identification scenarios, face recognition scenarios, product retrieval scenarios, landmark retrieval scenarios, clothing retrieval scenarios are only examples, and other scenarios applicable to the present invention should also be included within the protection scope of the present invention.

[0067] Step S202: Determine the scenario to which the target image belongs from several scenarios as the target scenario.

[0068] Exemplarily, the several scenarios include person re-identification scenarios, vehicle re-identification scenarios, face recognition scenarios, product retrieval scenarios, landmark retrieval scenarios, and clothing retrieval scenarios. The purpose of step S202 is to determine the scenario to which the target image belongs from these scenarios. Assuming the target image is a face image, it can be determined that the scenario to which the target image belongs is the face recognition scenario.

[0069] Step S203: Use the feature acquisition method corresponding to the target scenario among the feature acquisition methods corresponding to several scenarios to obtain the feature vector corresponding to the target image.

[0070] It should be noted that the feature acquisition method in this embodiment is a feature acquisition method constructed specifically for the specific attributes of images in the corresponding scenario (for example, the feature acquisition method corresponding to the face recognition scenario is a feature acquisition method constructed specifically for the face attributes of face images in the face recognition scenario). Through the feature acquisition method in this embodiment, a feature vector that can more finely represent the object in the image can be obtained. For example, using the feature acquisition method corresponding to the face recognition scenario can obtain a feature vector that can more finely represent the face in the face image.

[0071] Step S204: Based on the feature vector corresponding to the target image and the feature vectors corresponding to the respective images in the retrieval image library, determine the similarity between each image in the retrieval image library and the target image.

[0072] Specifically, the process of determining the similarity between each image in the retrieval image library and the target image based on the similarity between the feature vector corresponding to the target image and the feature vectors corresponding to the respective images in the retrieval image library may include: for each image in the retrieval image library, calculate the similarity between the feature vector corresponding to this image in the retrieval image library and the feature vector corresponding to the target image, and use it as the similarity between this image in the retrieval image library and the target image, so that the similarity between each image in the retrieval image library and the target image can be obtained.

[0073] Step S205: Determine the image retrieval result of the target image according to the similarity between each image in the retrieval image library and the target image.

[0074] There are various implementation manners for determining the image retrieval result of the target image according to the similarity between each image in the retrieval image library and the target image. For example, the images in the retrieval image library can be sorted in descending order of similarity to the target image, and the first N (N is an integer greater than or equal to 1, and the specific value of N can be set according to specific application scenarios) images can be used as the image retrieval result of the target image. Of course, the images in the retrieval image library can also be sorted in ascending order of similarity to the target image, and the last N images can be used as the image retrieval result of the target image. Another example is that images with a similarity to the target image greater than a preset similarity threshold can be screened from the retrieval image library, and the screened images can be used as the image retrieval result of the target image.

[0075] For the image retrieval method provided in the embodiments of the present invention, after obtaining the image to be retrieved, that is, the target image, first determine the scene to which the target image belongs from several scenes as the target scene, then obtain the feature vector corresponding to the target image based on the feature acquisition method corresponding to the target scene among the feature acquisition methods corresponding to the respective scenes, then based on the feature vector corresponding to the target image and the feature vectors corresponding to the respective images in the retrieval image library, determine the similarity between each image in the retrieval image library and the target image, and finally determine the image retrieval result corresponding to the target image according to the determined similarity. The image retrieval method provided in the embodiments of the present invention is applicable to multiple scenes, and since the feature vector corresponding to the image is obtained by using the feature acquisition method corresponding to the scene to which the image belongs, the obtained feature vector is a feature vector that can finely characterize the object in the image. Based on such a feature vector, fine-grained image retrieval can be realized, and finally a relatively accurate image retrieval result can be obtained. It can be seen that the image retrieval method provided in the embodiments of the present invention has a wide application range and high retrieval accuracy, and the user experience is good.

[0076] Second Embodiment

[0077] Based on the above embodiments, a specific example of the image retrieval method is provided in this embodiment. Please refer to Figure 3 , which shows a schematic flowchart of a specific example of the image retrieval method provided in this embodiment, and may include:

[0078] Step S301: Obtain the image to be retrieved as the target image.

[0079] Among them, the image to be retrieved can be an image of any one of several scenarios.

[0080] Step S302: Based on the pre-trained scene classification model, determine the scene to which the target image belongs from several scenarios as the target scene.

[0081] Among them, the scene classification model is trained using training images labeled with scene categories of several scenarios. When training the scene classification model, the training is carried out with the goal of making the scene category predicted by the scene classification model for the training image tend to be consistent with the scene category labeled on the training image.

[0082] Exemplarily, the scene classification model can be trained using training images labeled with scene categories of 6 scenarios including pedestrian re-identification scene, vehicle re-identification scene, face recognition scene, commodity retrieval scene, landmark retrieval scene, and clothing retrieval scene. After training, given an image of any one of the above 6 scenarios, the scene classification model can determine the scene to which the given image belongs from the above 6 scenarios.

[0083] Specifically, the process of determining the scene to which the target image belongs from several scenarios based on the pre-trained scene classification model may include: predicting the probabilities of the scene to which the target image belongs as each of the several scenarios based on the pre-trained scene classification model; determining the scene to which the target image belongs according to the probabilities of the scene to which the target image belongs as each of the several scenarios.

[0084] Step S303: Use the feature extraction model corresponding to the target scene among the feature extraction models corresponding to several scenarios to extract a feature vector from the target image, and the extracted feature vector is used as the feature vector corresponding to the target image.

[0085] Among them, the feature extraction model is trained using training images labeled with object identifiers corresponding to the scene. For example, the feature extraction model corresponding to the pedestrian re-identification scene is trained using training images labeled with pedestrian identifiers of the pedestrian re-identification scene, and the feature extraction model corresponding to the vehicle re-identification scene is trained using training images labeled with vehicle identifiers of the vehicle re-identification scene.

[0086] When training a feature extraction model corresponding to a target scenario using training images of the target scenario, the training is carried out with the goal of making the object identification predicted based on the feature vectors extracted from the training images tend to be consistent with the object identification annotated in the training images. Preferably, in order to obtain a feature extraction model with better performance, the training can also be carried out in combination with the following goals: making the feature vectors extracted from the training images close to the feature vectors extracted from similar samples (similar samples refer to other training images with the same object identification as the training images), and making the feature vectors extracted from the training images far from the feature vectors extracted from dissimilar samples (training images with different object identifications from the training images).

[0087] Since the feature extraction model in this embodiment is trained using training images of the corresponding scenario, therefore, it can extract feature vectors from the images of the corresponding scenario that can more finely characterize the objects in the images. For example, a feature extraction model trained using training images of the face recognition scenario can extract feature vectors from face images that can more finely characterize the faces in the face images.

[0088] Step S304: Based on the feature vector corresponding to the target image and the feature vectors corresponding to each image in the retrieval image library, determine the similarity between each image in the retrieval image library and the target image.

[0089] It should be noted that the method for obtaining the feature vectors corresponding to each image in the retrieval image library is similar to the method for obtaining the feature vector corresponding to the target image, that is, for each image in the retrieval image library, based on a pre-trained scene classification model, determine the scene to which the image belongs from several scenes, and use the feature extraction model corresponding to the scene to which the image belongs among the feature extraction models corresponding to several scenes to extract the feature vector from the image, and the extracted feature vector is used as the feature vector corresponding to the image.

[0090] For a more specific implementation process of step S304, reference can be made to the specific implementation process of step S204 in the above embodiment, and this embodiment will not be elaborated here.

[0091] Step S305: According to the similarity between each image in the retrieval image library and the target image, determine the image retrieval result of the target image.

[0092] For a more specific implementation process of step S305, reference can be made to the specific implementation process of step S205 in the above embodiment, and this embodiment will not be elaborated here.

[0093] The image retrieval method provided by the embodiments of the present invention, after obtaining the image to be retrieved, that is, the target image, first determines the scene to which the target image belongs from several scenes as the target scene based on the pre-trained scene classification model, and then extracts the feature vector of the target image as the feature vector corresponding to the target image based on the feature extraction model corresponding to the target scene among the feature extraction models corresponding to several scenes respectively. Then, based on the feature vector corresponding to the target image and the feature vectors corresponding to each image in the retrieval image library, the similarity between each image in the retrieval image library and the target image is determined. Finally, the image retrieval result corresponding to the target image is determined according to the determined similarity. The image retrieval method provided by the embodiments of the present invention can be applied to multiple scenes, and since the feature extraction model corresponding to the scene to which the image belongs is used to obtain the feature vector corresponding to the image, and the feature extraction model is trained with the training images of the corresponding scene, it can extract the feature vector that can more finely represent the object in the image for the image of the corresponding scene. Based on such a feature vector, fine-grained image retrieval can be realized, and finally a relatively accurate image retrieval result can be obtained. It can be seen that the image retrieval method provided by the embodiments of the present invention has a wide application range and high retrieval accuracy, and the user experience is good.

[0094] Third Embodiment

[0095] Considering that the feature extraction models corresponding to several scenes are trained separately, there may be a situation where the feature vector extracted by the feature extraction model corresponding to scene A for image a in scene A is very similar to the feature vector extracted by the feature extraction model corresponding to scene B for image b in scene B in some cases. If this situation occurs, there is a high probability that irrelevant images will appear in the retrieval results. In order to still be able to obtain a relatively accurate retrieval result when the feature vectors extracted by the feature extraction models corresponding to different scenes are similar, another specific example of the image retrieval method is provided in this embodiment. Please refer to Figure 4 , which shows the schematic flow diagram of another specific example of the image retrieval method provided in this embodiment, and may include:

[0096] Step S401: Obtain the image to be retrieved as the target image.

[0097] Among them, the image to be retrieved can be an image of any one of several scenes.

[0098] Step S402: Based on the pre-trained scene classification model, determine the scene to which the target image belongs from several scenes as the target scene.

[0099] For the specific implementation process and related explanations of step S402, reference can be made to the specific implementation process and related explanations of step S302 in the above embodiments, which will not be elaborated in this embodiment.

[0100] Step S403: Use the feature extraction model corresponding to the target scenario among the feature extraction models corresponding to several scenarios to extract feature vectors from the target image.

[0101] For the specific implementation process and related explanations of step S403, please refer to the specific implementation process and related explanations of step S303 in the above embodiments, which will not be elaborated herein.

[0102] Step S404: Perform attenuation processing on the eigenvalues at the processing positions corresponding to the target scenario in the extracted feature vectors, and use the feature vectors after attenuation processing as the feature vectors corresponding to the target image.

[0103] In a possible implementation, the process of performing attenuation processing on the eigenvalues at the processing positions corresponding to the target scenario in the extracted feature vectors may include: weighting the eigenvalues at the processing positions corresponding to the target scenario in the extracted feature vectors based on a preset weight. Among them, the preset weight is greater than 0 and less than 1, and the weight determines the attenuation amplitude of the eigenvalues at the processing positions corresponding to the target scenario in the extracted feature vectors. The specific value of the weight can be set according to specific circumstances.

[0104] It should be noted that the dimensions of the feature vectors extracted based on the feature extraction models corresponding to several scenarios are the same, and the processing positions corresponding to different scenarios are different. Exemplarily, there are 7 scenarios. The dimensions of the feature vectors extracted by the feature extraction model corresponding to the 1st scenario for the images of the 1st scenario, the dimensions of the feature vectors extracted by the feature extraction model corresponding to the 2nd scenario for the images of the 2nd scenario,..., and the dimensions of the feature vectors extracted by the feature extraction model corresponding to the 7th scenario for the images of the 7th scenario are all the same. The processing positions corresponding to the 7 scenarios are different. Assuming that the dimensions of the feature vectors extracted by the feature extraction models corresponding to the 7 scenarios are all 7, then the processing position corresponding to the 1st scenario is the 1st position of the feature vector, the processing position corresponding to the 2nd scenario is the 2nd position of the feature vector, the processing position corresponding to the 3rd scenario is the 3rd position of the feature vector,..., and the processing position corresponding to the 7th scenario is the 7th position of the feature vector.

[0105] The processing position corresponding to the target scenario is determined according to the target scenario, the total number of scenarios, and the dimension of the extracted feature vectors. Suppose there are N scenarios, the dimensions of the feature vectors extracted by the feature extraction models corresponding to the N scenarios are all M, the preset weight is α, and the target scenario is the nth scenario. The feature vector extracted from the target image using the feature extraction model corresponding to the nth scenario is f n , then the eigenvalue at the processing position corresponding to the nth scenario in f n is the eigenvalue from the ((n - 1)*P)th position to the (n*P)th position of the feature vector. For f nPerform attenuation processing on the eigenvalues from the (n - 1)*P-th bit to the n*P-th bit, that is, multiply f n the eigenvalues from the (n - 1)*P-th bit to the n*P-th bit in n by the weight α. P is an integer in the range (0, M / N], and the specific value of P can be set according to specific circumstances.

[0106] Exemplarily, there are 7 scenarios. The dimensions of the feature vectors extracted by the feature extraction models corresponding to the 7 scenarios are all 2048, and the preset weight is 0.1: If the target image is an image of the 1st scenario, and the feature vector extracted from the target image by the feature extraction model corresponding to the 1st scenario is f1, then the eigenvalues at the processing positions corresponding to the 1st scenario in f1 are the eigenvalues from the 0*256-th bit to the 1*256-th bit. Perform attenuation processing on the eigenvalues from the 0*256-th bit to the 1*256-th bit in f1, that is, multiply the eigenvalues from the 0*256-th bit to the 1*256-th bit in f1 by the weight 0.1; If the target image is an image of the 2nd scenario, and the feature vector extracted from the target image by the feature extraction model corresponding to the 2nd scenario is f2, then the eigenvalues at the processing positions corresponding to the 2nd scenario in f2 are the eigenvalues from the 1*256-th bit to the 2*256-th bit. Perform attenuation processing on the eigenvalues from the 1*256-th bit to the 2*256-th bit in f2, that is, multiply the eigenvalues from the 1*256-th bit to the 2*256-th bit in f2 by the weight 0.1; …; If the target image is an image of the 7th scenario, and the feature vector extracted from the target image by the feature extraction model corresponding to the 7th scenario is f7, then the eigenvalues at the processing positions corresponding to the 7th scenario in f7 are the eigenvalues from the 6*256-th bit to the 7*256-th bit. Perform attenuation processing on the eigenvalues from the 6*256-th bit to the 7*256-th bit in f7, that is, multiply the eigenvalues from the 6*256-th bit to the 7*256-th bit in f7 by the weight 0.1. It should be noted that the above 256 is an integer in (0, 2048 / 7].

[0107] Step S405: Determine the similarity between each image in the retrieval image library and the target image based on the similarity between the feature vector corresponding to the target image and the feature vectors corresponding to the images in the retrieval image library.

[0108] It should be noted that the feature vectors corresponding to the respective images in the retrieval image library can be obtained in advance. The method for obtaining the feature vectors corresponding to the respective images in the retrieval image library is similar to the method for obtaining the feature vector corresponding to the target image, that is, for each image in the retrieval image library, based on the scene classification model pre-trained, the scene to which the image belongs is determined from several scenes, and among the feature extraction models corresponding to the several scenes respectively, the feature extraction model corresponding to the scene to which the image belongs is used to extract the feature vector of the image, and the feature value at the processing position corresponding to the scene to which the image belongs in the extracted feature vector is attenuated, and the feature vector after attenuation is used as the feature vector corresponding to the image.

[0109] It should be noted that during the image retrieval process, it is necessary to determine the similarity between the target image and each image in the retrieval image library. Assume that the target image is image A and an image in the retrieval image library is image B:

[0110] When image A and image B belong to the same scene, the similarity determined in the case of using the feature vector after attenuation as the feature vector corresponding to the image is higher than the similarity determined in the case of using the non-attenuated feature vector (the non-attenuated feature vector refers to the feature vector directly extracted from the image by the feature extraction model) as the feature vector corresponding to the image. For example, in the case of using the non-attenuated feature vector as the feature vector corresponding to the image (using the feature vector extracted from image A by the feature extraction model corresponding to the scene to which image A belongs as the feature vector corresponding to image A, and using the feature vector extracted from image B by the feature extraction model corresponding to the scene to which image B belongs as the feature vector corresponding to image B), the similarity between the feature vector corresponding to image A and the feature vector corresponding to image B is 0.74664487, while in the case of using the feature vector after attenuation as the feature vector corresponding to the image (using the vector obtained by attenuating the feature vector extracted from image A by the feature extraction model corresponding to the scene to which image A belongs as the feature vector corresponding to image A, and using the vector obtained by attenuating the feature vector extracted from image B by the feature extraction model corresponding to the scene to which image B belongs as the feature vector corresponding to image B), the similarity between the feature vector corresponding to image A and the feature vector corresponding to image B is 0.74669978.

[0111] When Image A and Image B belong to different scenes, the similarity determined using the attenuated feature vector as the feature vector corresponding to the image is lower than that determined using the unattenuated feature vector as the feature vector corresponding to the image. For example, when using the unattenuated feature vector as the feature vector corresponding to the image, the similarity between the feature vector corresponding to Image A and the feature vector corresponding to Image B is 0.74912766, while when using the attenuated feature vector as the feature vector corresponding to the image, the similarity between the feature vector corresponding to Image A and the feature vector corresponding to Image B is 0.71081063. It can be seen that compared with the similarity determined using the unattenuated feature vector as the feature vector corresponding to the image, the similarity determined using the attenuated feature vector as the feature vector corresponding to the image has a significant decrease.

[0112] Attenuating the extracted feature vector has little impact on the similarity of images in the same scene, but it will significantly reduce the similarity of images in different scenes, so that a more accurate retrieval result can be obtained finally.

[0113] For a more specific implementation process of step S405, reference can be made to the specific implementation process of step S204 in the above embodiment, which will not be elaborated here in this embodiment.

[0114] Step S406: Determine the image retrieval result of the target image according to the similarity between each image in the retrieval image library and the target image.

[0115] For a more specific implementation process of step S406, reference can be made to the specific implementation process of step S205 in the above embodiment, which will not be elaborated here in this embodiment.

[0116] The image retrieval method provided by the embodiment of the present invention, after obtaining the image to be retrieved, that is, the target image, first determines the scene to which the target image belongs from several scenes as the target scene based on a pre-trained scene classification model, and then extracts the feature vector of the target image based on the feature extraction model corresponding to the target scene among the feature extraction models corresponding to several scenes respectively. Then, for the extracted feature vector, the feature value at the processing position corresponding to the target scene is attenuated, and the processed feature vector is used as the feature vector corresponding to the target image. Then, based on the feature vector corresponding to the target image and the feature vectors corresponding to each image in the retrieval image library respectively, the similarity between each image in the retrieval image library and the target image is determined. Finally, the image retrieval result corresponding to the target image is determined according to the determined similarity.

[0117] The image retrieval method provided by the embodiments of the present invention is applicable to multiple scenarios. Since the feature extraction model corresponding to the scene to which the image belongs is used to extract the feature vector of the image, and the feature extraction model is trained with the training images of the corresponding scene, it can extract the feature vector that can more finely represent the object in the image for the image of the corresponding scene. Based on such a feature vector, fine-grained image retrieval can be realized, and finally a more accurate image retrieval result can be obtained. Considering that the feature extraction models corresponding to several scenarios are trained separately, there may be a situation where the feature vectors extracted by the feature extraction models corresponding to different scenarios are similar at some times. When this situation occurs, in order to finally obtain a more accurate retrieval result, the embodiments of the present invention perform attenuation processing on the feature values at the processing positions corresponding to the scene to which the image belongs in the extracted feature vector, and use the feature vector after attenuation processing as the feature vector corresponding to the image. Performing attenuation processing on the extracted feature vector will greatly reduce the similarity of images in different scenarios, so that a more accurate retrieval result can be finally obtained. The image retrieval method provided by the embodiments of the present invention has a wide application range and high retrieval accuracy, and the user experience is good.

[0118] Fourth Embodiment

[0119] This embodiment provides an image similarity determination method, which is applicable to any application scenario that needs to calculate the similarity of images in multiple scenarios. Please refer to Figure 5 , which shows a schematic flowchart of the image similarity determination method, and may include:

[0120] Step S501: Obtain a first image.

[0121] Among them, the first image may be an image of any one of several scenarios.

[0122] Step S502: Determine the scene to which the first image belongs from several scenarios as the target scene.

[0123] Specifically, based on the pre-trained scene classification model, the scene to which the first image belongs can be determined from several scenarios as the target scene. Among them, the scene classification model is trained with the training images labeled with scene categories of several scenarios.

[0124] Step S503: Based on the feature acquisition method corresponding to the target scene among the feature acquisition methods corresponding to several scenarios respectively, obtain the feature vector corresponding to the first image.

[0125] Based on the feature acquisition method corresponding to the target scenario, there are multiple ways to obtain the feature vector corresponding to the first image: In one possible implementation, the feature vector can be extracted from the first image based on the feature extraction model corresponding to the target scenario, and the extracted feature vector is used as the feature vector corresponding to the first image; In another possible implementation, the feature vector can be first extracted from the first image based on the feature extraction model corresponding to the target scenario, and then the feature values at the processing positions corresponding to the target scenario in the extracted feature vector are attenuated, and the attenuated feature vector is used as the feature vector corresponding to the first image. In order to finally obtain a more accurate similarity, the second implementation is preferably used in this embodiment. Among them, the feature extraction model is trained using training images marked with object identifiers in the corresponding scenario.

[0126] For the specific implementation process of attenuating the feature values at the processing positions corresponding to the target scenario in the extracted feature vector, reference can be made to the relevant parts in the above embodiments, and this embodiment will not elaborate here.

[0127] Step S504: Determine the similarity between the first image and the second image based on the feature vector corresponding to the first image and the image feature vector corresponding to the second image.

[0128] Specifically, calculate the similarity between the feature vector corresponding to the first image and the image feature vector corresponding to the second image, and the calculated similarity is used as the similarity between the first image and the second image.

[0129] Among them, the image feature vector corresponding to the second image is obtained based on the feature acquisition method corresponding to the scenario to which the second image belongs. The image feature vector corresponding to the second image can be obtained in advance, or can be obtained when determining the similarity. Specifically, when to obtain it can be determined according to the specific application scenario.

[0130] The process of obtaining the image feature vector corresponding to the second image based on the feature acquisition method corresponding to the scenario to which the second image belongs is similar to the process of obtaining the feature vector corresponding to the first image based on the feature acquisition method corresponding to the target scenario, that is, the feature vector can be extracted from the second image based on the feature extraction model corresponding to the scenario to which the second image belongs, and the extracted feature vector is used as the image feature vector corresponding to the second image. Or the feature vector can be first extracted from the second image based on the feature extraction model corresponding to the scenario to which the second image belongs, and then the feature values at the processing positions corresponding to the scenario to which the second image belongs in the extracted feature vector are attenuated, and the attenuated feature vector is used as the image feature vector corresponding to the second image.

[0131] Among them, the dimensions of the feature vectors extracted based on the feature extraction models corresponding to the respective scenarios are the same, and the processing positions corresponding to different scenarios are different.

[0132] It should be noted that if the feature vector extracted from the first image is used as the feature vector corresponding to the first image, and the feature vector extracted from the second image is also used as the feature vector corresponding to the second image, or if the feature vector obtained after attenuating the feature vector extracted from the first image is used as the feature vector corresponding to the first image, then the feature vector obtained after attenuating the feature vector extracted from the second image is also used as the feature vector corresponding to the second image.

[0133] Through the image similarity determination method provided by the embodiments of the present invention, the similarity between the first image and the second image can be accurately determined.

[0134] Fifth Embodiment

[0135] The embodiments of the present invention also provide an image retrieval device. The image retrieval device provided by the embodiments of the present invention will be described below. The image retrieval device described below can be correspondingly referred to the image retrieval method described above.

[0136] Please refer to Figure 6 , which shows a schematic structural diagram of the image retrieval device provided by the embodiments of the present invention, and may include: an image acquisition module 601, a scene determination module 602, a feature acquisition module 603, a similarity determination module 604, and a retrieval result determination module 605.

[0137] The image acquisition module 601 is configured to acquire an image to be retrieved as a target image.

[0138] The scene determination module 602 is configured to determine the scene to which the target image belongs from several scenes as the target scene.

[0139] The feature acquisition module 603 is configured to acquire the feature vector corresponding to the target image based on the feature acquisition method corresponding to the target scene among the feature acquisition methods corresponding to several scenes respectively.

[0140] The similarity determination module 604 is configured to determine the similarity between each image in the retrieval image library and the target image based on the feature vector corresponding to the target image and the feature vectors corresponding to the respective images in the retrieval image library.

[0141] Among them, the feature vector corresponding to an image in the retrieval image library is obtained based on the feature acquisition method corresponding to the scene to which the image belongs.

[0142] The retrieval result determination module 605 is configured to determine the image retrieval result corresponding to the target image according to the similarity between each image in the retrieval image library and the target image.

[0143] Optionally, the feature acquisition module 603 may include: a feature extraction module and a feature processing module.

[0144] A feature extraction module, configured to extract a feature vector from the target image based on the feature extraction model corresponding to the target scene among the feature extraction models corresponding to the several scenes respectively.

[0145] Wherein, the feature extraction model is trained by using training images corresponding to the corresponding scene;

[0146] A feature processing module, configured to perform attenuation processing on the feature value at the processing position corresponding to the target scene in the feature vector extracted by the feature extraction module, and the feature vector after the attenuation processing is used as the feature vector corresponding to the target image.

[0147] Wherein, the dimensions of the feature vectors extracted based on the feature extraction models corresponding to the several scenes respectively are the same, and the processing positions corresponding to different scenes are different.

[0148] Optionally, the processing position corresponding to the target scene is determined according to the target scene, the total number of scenes of the several scenes, and the dimension of the feature vector extracted from the target image based on the feature extraction model corresponding to the target scene.

[0149] Optionally, when the feature processing module performs attenuation processing on the feature value at the processing position corresponding to the target scene in the extracted feature vector, it is specifically configured to:

[0150] Weight the feature value at the processing position corresponding to the target scene in the extracted feature vector based on a preset weight, where the weight is greater than 0 and less than 1.

[0151] Optionally, when the scene determination module 602 determines the scene to which the target image belongs from several scenes, it is specifically configured to:

[0152] Determine the scene to which the target image belongs based on a pre-trained scene classification model; wherein, the scene classification model is trained by using training images labeled with scene categories of several scenes.

[0153] The image retrieval device provided by the embodiment of the present invention can be applied to multiple scenes, and since the feature vector corresponding to the image is obtained by using the feature acquisition method corresponding to the scene to which the image belongs, the obtained feature vector is a feature vector that can finely represent the object in the image. Based on such a feature vector, fine-grained image retrieval can be realized, and finally a relatively accurate image retrieval result can be obtained. It can be seen that the image retrieval device provided by the embodiment of the present invention has a wide application range and high retrieval accuracy, and the user experience is good.

[0154] Sixth Embodiment

[0155] The embodiment of the present invention also provides an image retrieval device, please refer to Figure 7, showing a schematic structural diagram of the image retrieval device, which may include: at least one processor 701, at least one communication interface 702, at least one memory 703, and at least one communication bus 704;

[0156] In the embodiments of the present invention, the number of the processor 701, the communication interface 702, the memory 703, and the communication bus 704 is at least one, and the processor 701, the communication interface 702, and the memory 703 complete mutual communication through the communication bus 704;

[0157] The processor 701 may be a central processing unit CPU, or a specific integrated circuit ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of the present invention, etc.;

[0158] The memory 703 may include a high-speed RAM memory, and may also include a non-volatile memory, etc., such as at least one disk memory;

[0159] Wherein, the memory stores a program, and the processor can call the program stored in the memory, and the program is used for:

[0160] Obtain the image to be retrieved as the target image;

[0161] Determine the scene to which the target image belongs from several scenes as the target scene;

[0162] Based on the feature acquisition method corresponding to the target scene among the feature acquisition methods corresponding to the several scenes respectively, obtain the feature vector corresponding to the target image;

[0163] Based on the feature vector corresponding to the target image and the feature vectors corresponding to each image in the retrieval image library respectively, determine the similarity between each image in the retrieval image library and the target image; wherein, the feature vector corresponding to an image in the retrieval image library is obtained based on the feature acquisition method corresponding to the scene to which the image belongs; [[ID=2,6]]

[0164] According to the determined similarity, determine the image retrieval result corresponding to the target image.

[0165] Optionally, the refined functions and extended functions of the program may refer to the above description.

[0166] Seventh Embodiment

[0167] The embodiments of the present invention also provide a readable storage medium, which can store a program suitable for being executed by a processor, and the program is used for:

[0168] Obtain the image to be retrieved as the target image;

[0169] Determine the scene to which the target image belongs from several scenes as the target scene;

[0170] Based on the feature acquisition method corresponding to the target scene among the feature acquisition methods corresponding to the several scenes respectively, obtain the feature vector corresponding to the target image;

[0171] Based on the feature vector corresponding to the target image and the feature vectors corresponding to each image in the retrieval image library respectively, determine the similarity between each image in the retrieval image library and the target image; wherein, the feature vector corresponding to an image in the retrieval image library is obtained based on the feature acquisition method corresponding to the scene to which the image belongs;

[0172] According to the determined similarity, determine the image retrieval result corresponding to the target image.

[0173] Optionally, the refined functions and extended functions of the program can be referred to the above description.

[0174] Finally, it should also be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprises", "comprising" or any other variation thereof is intended to cover a non-exclusive inclusion, such that a process, method, article or device comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or device. Without further limitation, an element defined by the phrase "comprising a..." does not exclude the presence of additional identical elements in the process, method, article or device comprising the element.

[0175] Each embodiment in this specification is described in a progressive manner. The key point of each embodiment is to illustrate the differences from other embodiments. The same or similar parts among the embodiments can be referred to each other.

[0176] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present invention. Various modifications to these embodiments will be obvious to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but will be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An image retrieval method, characterized in that, Including: Obtain the image to be retrieved as the target image; Determine the scene to which the target image belongs from several scenes as the target scene; Based on the feature extraction model corresponding to the target scene among the feature extraction models corresponding to the several scenes respectively, extract a feature vector from the target image; wherein, the feature extraction model is trained using training images of the corresponding scene; Perform attenuation processing on the feature value at the processing position corresponding to the target scene in the extracted feature vector to reduce the similarity of images in different scenes, and the feature vector after attenuation processing is used as the feature vector corresponding to the target image; wherein, the dimensions of the feature vectors extracted based on the feature extraction models corresponding to the several scenes respectively are the same, and the processing positions corresponding to different scenes are different; Based on the feature vector corresponding to the target image and the feature vectors corresponding to the respective images in the retrieval image library, determine the similarity between each image in the retrieval image library and the target image; wherein, the feature vector corresponding to an image in the retrieval image library is obtained based on the feature acquisition method corresponding to the scene to which the image belongs; According to the determined similarity, determine the image retrieval result corresponding to the target image.

2. The image retrieval method according to claim 1, characterized in that The processing position corresponding to the target scene is determined according to the target scene, the total number of scenes of the several scenes, and the dimension of the feature vector extracted from the target image based on the feature extraction model corresponding to the target scene.

3. The image retrieval method according to claim 1, wherein The performing attenuation processing on the feature value at the processing position corresponding to the target scene in the extracted feature vector includes: Based on a preset weight, weight the feature value at the processing position corresponding to the target scene in the extracted feature vector, wherein the weight is greater than 0 and less than 1.

4. The image retrieval method according to claim 1, characterized in that The determining the scene to which the target image belongs from several scenes includes: Based on a pre-trained scene classification model, determine the scene to which the target image belongs; wherein, the scene classification model is trained using training images with labeled scene categories of several scenes.

5. A method for determining image similarity, characterized in that, Including: Obtain a first image; Determine the scene to which the first image belongs from several scenes as the target scene; Based on the feature extraction model corresponding to the target scene among the feature extraction models corresponding to the several scenes respectively, extract a feature vector from the first image; wherein, the feature extraction model is trained using training images of the corresponding scene; Perform attenuation processing on the feature value at the processing position corresponding to the target scene in the extracted feature vector to reduce the similarity of images in different scenes, and the feature vector after attenuation processing is used as the feature vector corresponding to the first image; wherein, the dimensions of the feature vectors extracted based on the feature extraction models corresponding to the several scenes respectively are the same, and the processing positions corresponding to different scenes are different; Based on the feature vector corresponding to the first image and the image feature vector corresponding to the second image, determine the similarity between the first image and the second image; wherein, the image feature vector corresponding to the second image is obtained based on the feature acquisition method corresponding to the scene to which the second image belongs.

6. An image retrieval device, characterized in that, Including: An image acquisition module, a scene determination module, a feature acquisition module, a similarity determination module, and a retrieval result determination module; The image acquisition module is configured to acquire an image to be retrieved as a target image; The scene determination module is configured to determine the scene to which the target image belongs from a plurality of scenes as a target scene; The feature acquisition module includes a feature extraction module and a feature processing module; The feature extraction module is configured to extract a feature vector from the target image based on the feature extraction model corresponding to the target scene among the feature extraction models corresponding to the plurality of scenes respectively, wherein the feature extraction model is trained with training images corresponding to the corresponding scene; The feature processing module is configured to perform attenuation processing on the feature values at the processing positions corresponding to the target scene in the feature vector extracted by the feature extraction module to reduce the similarity of images in different scenes, and the feature vector after the attenuation processing is used as the feature vector corresponding to the target image, wherein the dimensions of the feature vectors extracted based on the feature extraction models corresponding to the plurality of scenes respectively are the same, and the processing positions corresponding to different scenes are different; The similarity determination module is configured to determine the similarity between each image in the retrieval image library and the target image based on the feature vector corresponding to the target image and the feature vectors corresponding to each image in the retrieval image library respectively, wherein the feature vector corresponding to an image in the retrieval image library is obtained based on the feature acquisition method corresponding to the scene to which the image belongs; The retrieval result determination module is configured to determine the image retrieval result corresponding to the target image according to the similarity between each image in the retrieval image library and the target image.

7. An image retrieval device, characterized in that, Including: A memory and a processor; The memory is configured to store a program; The processor is configured to execute the program to implement each step of the image retrieval method according to any one of claims 1 to 4.

8. A readable storage medium, on which a computer program is stored, characterized in that, When the computer program is executed by the processor, each step of the image retrieval method according to any one of claims 1 to 4 is implemented.

Images