An image search reordering system

Abstract

The application provides an image search reordering system, which sets a first image library belonging to each object in an image search library and a second image library not belonging to the object, and if the maximum similarity between a target image and an image in the second image library is greater than the maximum similarity between the target image and an image in the first image library when searching the closest image based on the target image, the maximum similarity between the target image and the image in the first image library is corrected based on the maximum similarity between the target image and the image in the second image library, so that the maximum similarity between the target image and the image in the first image library can be reduced, and further, inaccurate images can be avoided, and the search accuracy can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of picture information processing, and particularly to an image search reordering system. BACKGROUND

[0002] The current image search method is that when a user inputs a target image to be queried, the feature information of the target image is first extracted to obtain a high-dimensional feature vector, then the similarity between the high-dimensional feature vector and the high-dimensional feature vector of each image in an image database is calculated, the image with the highest similarity is taken as the image closest to the target image, so as to determine the identity of the target image. However, when there are images highly similar to the target image but not belonging to the same object, the closest image given is not necessarily accurate. SUMMARY

[0003] In view of the above technical problems, the technical scheme adopted by the present application is that:

[0004] The embodiment of the present application provides an image search reordering system, which comprises a processor and an image search library connected in communication, wherein the image search library comprises n image libraries corresponding to n objects, the image library i comprises a first image library I i1 and a second image library I i2 , wherein I i1 =(I 1 i1 , I 2 i1 , …, I j i1 , …, I m1(i) i1 ), I j i1 is the jth image in I i1 , I j i1 is the image of the object i, and j is 1 to m1(i), wherein m1(i) is the number of images in I i1 , I i2 =(I 1 i2 , I 2 i2 , …, I k i2 , …, I m2(i) i2 ), I k i2 is the kth image in I i2 , I k i2 is the image not of the object i, and k is 1 to m2(i), wherein m2(i) is the number of images in I i2The number of images in the dataset; i ranges from 1 to n;

[0005] Upon receiving the target image P, the processor executes a computer program to perform the following steps:

[0006] S100: Traverse the image search library. For each image in the library i that has been traversed, if i ≤ n, execute S200; otherwise, execute S700.

[0007] S200, Obtain the first similarity set D P i1 =(D P1 i1 D P2 i1 D Pj i1 D Pm1(i) i1 ), D P i1 The j-th similarity D in Pj i1 For P and I j i1 The similarity between them;

[0008] S300, Obtain the second similarity set D P i2 =(D P1 i2 D P2 i2 D Pk i2 D Pm2(i) i2 ), D P i2 The kth similarity D in Pk i2 For P and I k i2 The similarity between them;

[0009] S400, obtain max(D) P i1 ) and max(D P i2 If max(D) P i2 )>max(D P i1 ), set S ip =f(max(D) P i1 ), max(D P i2 )) is the similarity between the image of object i and P, where f(max(D) is the similarity between the two images.P i1 ), max(D P i2 )) and max(D P i1 ) positively, and max(D P i2 ) negatively; otherwise, set S ip = max(D P i1 ) as the similarity between the image of object i and P;

[0010] S500, if S ip > D, store S ip to the intermediate similarity set D c , and the initial value of D c is an empty set; D is a set similarity threshold value;

[0011] S600, set i = i + 1; execute S100;

[0012] S700, based on S100-S500, obtain D c = (D 1 c , D 2 c , …, D g c , …, D h c ); D g c is the gth similarity in D c , g takes values from 1 to h, and h is the number of similarities in D g c ;

[0013] S800, sort D c in descending order of similarity to obtain the target similarity set D t = (D 1 t , D 2 t , …, D g t , …, D h t ), D g t is the gth similarity in D t ;

[0014] S900, based on D t , obtain the candidate image set I t = (I 1 t , I2 t ,..., I g t ,..., I h t ), I g t for D g t corresponding images in the first image library.

[0015] The present application has at least the following beneficial effects:

[0016] The image search reordering system provided by the embodiments of the present application can reduce the maximum similarity of the images in the first image library, and further avoid searching inaccurate images and improve the search accuracy, for each object in the image search library, because the first image library and the second image library not belonging to the object are set, when searching the closest images based on the target image, if the maximum similarity of the target image and the images in the second image library of a certain object is greater than the maximum similarity of the target image and the images in the first image library, the maximum similarity of the target image and the images in the first image library is corrected based on the maximum similarity of the target image and the images in the second image library. BRIEF DESCRIPTION OF DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiment description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0018] Figure 1 The structure diagram of the image search reordering system provided by the embodiments of the present application. DETAILED DESCRIPTION

[0019] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0020] Figure 1 The structure diagram of the image search reordering system provided by the embodiments of the present application. As shown in Figure 1 The present application provides an image search reordering system, which comprises a processor and an image search library in communication connection, the image search library comprises n image libraries corresponding to n objects, and the image library i comprises a first image library Ii1 and a second image library I i2 wherein I i1 = (I 1 i1 , I 2 i1 , …, I j i1 , …, I m1(i) i1 ), I j i1 is the jth image in I i1 , I j i1 is an image of object i, and j takes values from 1 to m1(i), where m1(i) is the number of images in I i1 ; I i2 = (I 1 i2 , I 2 i2 , …, I k i2 , …, I m2(i) i2 ), I k i2 is the kth image in I i2 , I k i2 is an image not of object i, and k takes values from 1 to m2(i), where m2(i) is the number of images in I i2 ; and i takes values from 1 to n.

[0021] In an embodiment of the present application, the object can be a human.

[0022] In an embodiment of the present application, the images in the first image library are images manually labeled, i.e., a plurality of images manually labeled as belonging to the same object. The images in the second image library are images manually labeled, i.e., the images in the second image library are images manually labeled as not belonging to the object corresponding to the first image library. The images in the second image library can include images of a plurality of objects.

[0023] In an embodiment of the present application, the images in the second image library can or can not be associated with the images in the first image library. In an embodiment of the present application, being associated means that the images are similar to each other. In an illustrative embodiment, some of the images in the second image library can be associated with the images in the first image library, for example, the similarity between some of the images in the second image library and the images in the first image library can be greater than a set threshold value.

[0024] In the embodiments of the present application, the system further comprises a storage medium storing a computer program. When the target image P is received, the processor is configured to execute the computer program to implement the following steps:

[0025] S100, traverse the image search library, if i≤n, execute S200; otherwise, execute S700.

[0026] S200, obtain the first similarity set D P i1 P1 i1 P2 i1 Pj i1 Pm1(i) i1 P i1 Pj i1 j i1

[0027] The target image P can be an image input by a user. In the embodiments of the present application, D P i1 may be obtained based on existing methods, for example, can comprise the following steps:

[0028] S201, extract the feature vector of the target image to form a target image feature vector list V P P P P r P u P r is the rth feature vector of the target image, and u is the number of feature vectors of the target image, for example, u=512. Those skilled in the art know that extracting the feature vector of the target image to form the target image feature vector list can be prior art, for example, extracted by a deep neural network.

[0029] S202, calculate the similarity D P i1 j i1 j i1 Pj i1 j ​​​​​​​​​​​​​​​​​​i1 = (V j1 i1 , V j2 i1 , …, V jr i1 , …, V ju i1 ), i.e., V j i1 is a vector with the same dimension as V P , V jr i1 is the rth eigenvector of I j i1 . V j i1 may be extracted in advance and stored in a database.

[0030] In the embodiments of the present application, the set similarity calculation method can be an existing method, for example, cosine distance, Euclidean distance, and Hamming distance, etc.

[0031] S300, obtaining a second similarity set D P i2 = (D P1 i2 , D P2 i2 , …, D Pk i2 , …, D Pm2(i) i2 ), D P i2 is the kth similarity D Pk i2 in D k i2 is the similarity between P and I P i1 .

[0032] The specific implementation of S300 can be the same as that of S200.

[0033] S400, obtaining max(D P i2 ) and max(D P i2 ), if max(D P i1 ) > max(D ip P ), setting S i1 = f(max(D P i2 ), max(D P i1), max(D P i2 )) and max(D P i1 Positive correlation, i.e., max(D) P i1 The larger S is, the better. ip The larger; and the greater the value of max(D) P i2 Negative correlation, i.e., max(D) P i2 The larger S is, the better. ip The smaller the value, the better; otherwise, set S. ip =max(D P i1 The similarity between the image of object i and P.

[0034] In a specific example of the present invention, f(max(D) P i1 ), max(D P i2 ))=λ1*max(D P i1 )-λ2*max(D P i2 ), where λ1 and λ2 are set coefficients, with values ​​ranging from 0 to 1. λ1 > λ2.

[0035] In another specific example of the invention, f(max(D) P i1 ), max(D P i2 ))=max(D P i1 )-λ*max(D P i2 ), where λ is a set coefficient, taking a value from 0 to 1. Compared to the previous embodiment, since S is calculated... ip When, max(D) P i1 The actual values ​​are used, and the corrected similarity will be more accurate than the aforementioned embodiments.

[0036] The technical advantage of S400 lies in the fact that if max(D) P i2 )>max(D P i1 This indicates that the target image is more similar to the image in the second image library than the image in the first image library. Therefore, the similarity calculated from the search results in the first image library needs to be corrected downwards to avoid the search results in the first image library appearing in the high-confidence matching results of the target image, thereby ensuring the accuracy of the search results.

[0037] S500, if S ip > D, set S ip to the intermediate similarity set D c , where the initial value of D c is an empty set; D is a set similarity threshold. In the embodiments of the present application, D can be determined based on the test data set, for example, D can be 0.85-0.95.

[0038] S600, set i=i+1; execute S100;

[0039] S700, based on S100-S500, obtain D c =(D 1 c , D 2 c , …, D g c , …, D h c ); D g c is the gth similarity in D c , g takes a value from 1 to h, and h is the number of similarities in D g c . Obviously, any D g c > D.

[0040] S800, arrange D c in descending order of similarity to obtain the target similarity set D t =(D 1 t , D 2 t , …, D g t , …, D h t ), where the gth similarity D t in D g t ∈ D c , obviously, D g t ≤ D g+1 t ≤ D g+2 t ;

[0041] S900, based on D t , obtain the candidate image set I t =(I 1 t , I 2 t , …, I gt , ..., I h t ), I g t D g t The corresponding images in the first image library, i.e., based on D t Each image is retrieved from each of the first image libraries in the image search library to form a candidate image set.

[0042] The image search re-ranking system provided in this embodiment of the invention, for each object in the image search library, since a first image library belonging to the object and a second image library not belonging to the object are set, when searching for the closest image based on the target image, if the maximum similarity between the target image and an image in the second image library of a certain object is greater than the maximum similarity with an image in the first image library, then the maximum similarity with the image in the first image library is corrected based on the maximum similarity with the image in the second image library. In this way, the maximum similarity with the image in the first image library can be reduced, and the image of the highly similar object can be avoided as the closest image when there are highly similar objects corresponding to the target image. That is, inaccurate images can be avoided and the search accuracy can be improved.

[0043] In this embodiment of the invention, λ (including λ1 and λ2) can be determined experimentally. In a specific embodiment of the invention, λ = Avg(λ1, λ2, ..., λ v , …, λ N ), λ v Let λ be the value obtained in the v-th trial, where v ranges from 1 to N, N is the number of trials, and λ is the value obtained in the v-th trial. v It was obtained in the following way:

[0044] (1) In the vth experiment, the target image v is input into the system, and W different λ values ​​are set respectively;

[0045] (2) Obtain W candidate image sets based on steps S100 to S900;

[0046] (3) Select the set of images that best matches the target image v from the W candidate image sets, and take the corresponding λ as λ. v .

[0047] In a specific example, preferably, λ = 0.5.

[0048] Furthermore, in another embodiment of the present invention, S900 is replaced by:

[0049] S910, Obtain candidate image set I t =(I 1 t I 2t I g t I s t I g t D g t corresponding image in the first image library, s

[0050] In S910, the images ranked in the front s are obtained as candidate images, compared with the foregoing embodiments, the number of candidate images can be reduced to facilitate user browsing of the visualization result. The value of s can be set by the user.

[0051] Further, in the embodiments of the present application, the system further comprises a display (not shown) connected to the processor in communication.

[0052] Further, in the embodiments of the present application, the processor is further configured to execute a computer program to implement the following steps:

[0053] S1000, displaying I t on the display.

[0054] By displaying each image in I t on the display in order of similarity, the user can intuitively know the search result.

[0055] Although some specific embodiments of the present application have been described in detail by examples, those skilled in the art should understand that the above examples are only for illustration, not for limiting the scope of the present application. Those skilled in the art should also understand that various modifications can be made to the embodiments without departing from the scope and spirit of the present application. The scope of the present application is defined by the appended claims.

Claims

1. An image search re-ranking system, characterized by, The system comprises a processor and an image search library connected in communication, the image search library comprises n image libraries corresponding to n objects, the image library i comprises a first image library I i1 and a second image library I i2 , wherein I i1 =(I 1 i1 , I 2 i1 , …, I j i1 , …, I m1(i) i1 ), I j i1 is the jth image in I i1 , I j i1 is the image of the object i, and j is 1 to m1(i), wherein m1(i) is the number of images in I i1 ; I i2 =(I 1 i2 , I 2 i2 , …, I k i2 , …, I m2 (i) i2 ), I k i2 is the kth image in I i2 , I k i2 is the image that is not the image of the object i, and k is 1 to m2(i), wherein m2(i) is the number of images in I i2 ; and i is 1 to n. When the target image P is received, the processor is configured to execute a computer program to implement the following steps: S100, traversing the image search library, if i≤n, executing S200 for the traversed image library i, otherwise, executing S700; S200, obtaining a first similarity set D P i1 = (D P1 i1 , D P2 i1 , …, D Pj i1 , …, D Pm1(i) i1 ), D P i1 jth similarity D Pj i1 between P and I j i1 ; S300, obtaining a second similarity set D P i2 = (D P1 i2 , D P2 i2 , …, D Pk i2 , …, D Pm2(i) i2 ), D P i2 kth similarity D Pk i2 between P and I k i2 ; S400, obtaining max(D P i1 ) and max(D P i2 ), if max(D P i2 ) > max(D P i1 ), setting S ip = f(max(D P i1 ), max(D P i2 )) as the similarity between the image of the object i and P, wherein f(max(D P i1 ), max(D P i2 )) is positively correlated with max(D P i1 ) and negatively correlated with max(D P i2 ); otherwise, setting S ip = max(D P i1 ) as the similarity between the image of the object i and P; S500, if S ip > D, S ip to the intermediate similarity set D c , D c The initial value of D is an empty set; D is a set similarity threshold value; S600, setting i=i+1; executing S100; S700, obtaining D based on S100-S500 c = (D 1 c , D 2 c , …, D g c , …, D h c ); D g c is the gth similarity in D c , g is valued from 1 to h, h is the number of similarities in D c . S800, D c are sorted in descending order of similarity size, obtaining a target similarity set D t = (D 1 t , D 2 t , …, D g t , …, D h t ), D g t is the gth similarity in D t ; S900, based on D t , a candidate image set I t 1 t , I 2 t , …, I g t , …, I h t ), I g t for D g t the image in the corresponding first image library.​ 2. The system of claim 1, wherein, f(max(D P i1 ), max(D P i2 )) = max(D P i1 )-λ*max(D P i2 ), λ is a set coefficient, and the value is 0 to 1.

3. The system of claim 1, wherein, S900 is replaced by: S910, obtaining a candidate image set I t = (I 1 t , I 2 t , …, I g t , …, I s t ), I g t for D g t corresponding to the first image in the image bank, s 4. The system of claim 1 or 3, wherein, The system further comprises a display connected in communication with the processor; The processor is further configured to execute a computer program to implement the following steps: S1000, on the display, display I t .

5. The system of claim 1, wherein, The images in the second image library comprise images of multiple objects.

6. The system of claim 1, wherein, The images in the first image library are manually annotated images.

7. The system of claim 1, wherein, The images in the second image library are manually annotated images.

Images