Combination search result optimization method and device, computer device, and storage medium

Abstract

The present application relates to the technical field of face picture search, and particularly relates to a combined search result optimization method and device, computer equipment and a storage medium. The method constructs a high-confidence search support set, updates the similarity of each picture in the high-confidence search support set to a comprehensive similarity relative to N to-be-retrieved face pictures, and updates the similarity of each picture in the remaining picture set to a comprehensive similarity relative to N to-be-retrieved face pictures and M high-confidence search support set pictures, thereby replacing the original similarity based on a single picture with a comprehensive similarity based on multiple pictures, fully considering the representation difference of face features between different pictures, and being able to ensure that the higher the comprehensive similarity, the greater the probability that the picture and the to-be-retrieved face picture belong to the same person, and finally arranging in descending order according to the updated similarity, which can realize that the pictures of the same person as the to-be-retrieved face picture are as far forward as possible, and the pictures of different persons are as far backward as possible.

Description

Technical Field

[0001] This invention relates to the field of facial image search technology, and in particular to a method, apparatus, computer device, and storage medium for optimizing combined search results. Background Technology

[0002] Currently, the collected facial images exhibit diversity. Due to shooting settings (such as wide angles or blurriness) or faces being obscured by masks, hats, glasses, etc., some facial features are often lost in the collected images. This poses a significant challenge to the effectiveness of facial image search based on facial feature similarity. For these reasons, current search results based on a single facial image are often incomplete, missing some images of the person in question. In such cases, a common approach is to search multiple facial images of the same person separately, and then combine the results from each image into a single search result to improve the completeness of the search results. While combined search results based on multiple facial images are more complete, a crucial issue remains: how to process these combined search results to ensure that images of the person in question appear as high as possible in the combined search results, while images of others appear as low as possible in the combined results.

[0003] The current mainstream approach to ranking combined search results of multiple face images is to first deduplicate the combined search results, retaining the highest similarity value during the deduplication process, and then sort all the deduplicated images according to their similarity. However, this approach, when sorting all deduplicated images by similarity, still calculates the similarity of each image based on a single face image. Because it ignores the differences in representation between images, the ranking effect is not ideal. For example, for a clear, unobstructed frontal face image, since the facial features are clearly and completely represented, when searching for this face image, the higher the similarity of a search result image, the greater the probability that it belongs to the same person as the searched face image. However, for an image where the face is heavily obscured by a mask or hat, most facial features are lost under the cover. Searching for this image yields two main types of results: First, there are similar images with similar levels of obscuration. Even if they are not the same person, the severe homogeneous occlusion makes their facial features very similar, leading to higher similarity between images of people wearing masks and hats. However, since the search focuses on homogeneous occlusion, the probability of these highly similar images belonging to the same person is usually very low. Second, there are unobscured images found based on only a small portion of the facial features remaining. These images typically have low similarity to the searched image, but because the search focuses on facial features, the probability of these low-similarity images belonging to the same person is actually higher. In other words, for heavily obscured images, higher similarity in search results does not necessarily mean a higher probability of belonging to the same person. One cannot rely solely on similarity to determine if an image represents the person in question.

[0004] In summary, although combined search results are obtained by searching multiple facial images of the same person, even images of the same person can differ in facial features due to shooting settings or occlusion of the face. The clearer and more complete the facial features in the image, the greater the probability that the image with the highest similarity in the search results is the person's image. Conversely, if the facial features are few or blurry, the probability of the image being the person's image with the highest similarity in the search results is lower. In other words, because of the differences in representation between images, judging whether an image is of the person based solely on the similarity calculated from a single image is sometimes unreliable. High similarity scores may not necessarily indicate the person is not the same, and low similarity scores may not necessarily indicate the person is not the same. Therefore, the current method of directly deduplicating and sorting combined search results does not achieve a good ranking result; it does not effectively ensure that images of the person are ranked as high as possible, and images of non-persons are ranked as low as possible. Summary of the Invention

[0005] In view of this, embodiments of the present invention provide a method, apparatus, computer device, and storage medium for optimizing combined search results, in order to solve the problem that the current combined search results of multiple face images are not sorted well, and it is impossible to make the images of the person being searched as high as possible and the images of other people being searched as low as possible.

[0006] In a first aspect, embodiments of the present invention provide a method for optimizing combined search results, the method comprising:

[0007] Obtain combined search results for N face images to be searched; the combined search results include N image sequences, each image sequence being K images obtained from the search of the corresponding single face image to be searched, arranged in descending order of similarity, where N and K are both integers greater than zero;

[0008] From the combined search results, M images belonging to the same person as the N face images to be retrieved are selected to form a high-confidence search support set, and the remaining images form the remaining image set, where 1≤M<N;

[0009] The similarity of N×K images in the combined search results is updated using the high-confidence search support set. For each image in the high-confidence search support set, the overall similarity of each image relative to N images in the search results is calculated based on the similarity between each image and a single image of the face to be retrieved, and this is used as the updated similarity. For the remaining images in the image set, the overall similarity of each image relative to N images of the face to be retrieved and M images in the high-confidence search support set is calculated based on the similarity between each image and a single image of the face to be retrieved and a single image in the high-confidence search support set, and this is used as the updated similarity.

[0010] The N×K images in the combined search results are sorted in descending order of their updated similarity to obtain the optimal ranking of the combined search results.

[0011] In one embodiment, selecting M images from the combined search results that belong to the same person as the N facial images to be retrieved to form a high-confidence search support set includes: for the N image sequences in the combined search results, truncating each image sequence using a preset similarity threshold to obtain N truncated sequences, retaining only images with a similarity greater than the similarity threshold in each truncated sequence, taking the intersection of the N truncated sequences, and using images that appear simultaneously in at least L truncated sequences as high-confidence images, and using all the high-confidence images to form a high-confidence search support set, where 2≤L≤N.

[0012] In one embodiment, selecting M images from the combined search results that belong to the same person as the N facial images to be retrieved to form a high-confidence search support set includes: for the N image sequences in the combined search results, truncating each image sequence using a preset similarity threshold to obtain N truncated sequences, retaining only images with a similarity greater than the similarity threshold in each truncated sequence, and taking the union of the N truncated sequences to obtain the high-confidence search support set.

[0013] In one embodiment, for images in the high-confidence search support set, calculating the comprehensive similarity of each image relative to N face images to be searched, based on the similarity between each image and a single face image to be searched, includes: for images in the high-confidence search support set, calculating the similarity between each image and N face images to be searched, and then performing statistical analysis on the N similarities of each image to obtain the comprehensive similarity of each image relative to N face images to be searched; for images in the remaining image set, based on the similarity between each image and a single face image to be searched, calculating the comprehensive similarity of each image relative to N face images to be searched; The similarity between the image of the face to be retrieved and a single high-confidence search support set image is calculated. The comprehensive similarity of each image relative to N images of the face to be retrieved and M images of the high-confidence search support set is calculated as follows: For the images in the remaining image set, the similarity between each image and N images of the face to be retrieved and M images of the high-confidence search support set is calculated separately. Then, the (N+M) similarity scores of each image are statistically analyzed to obtain the comprehensive similarity of each image relative to N images of the face to be retrieved and M images of the high-confidence search support set.

[0014] In one embodiment, the statistical analysis of the N similarities of each calculated image is performed by taking the arithmetic mean of the N similarities; the statistical analysis of the (N+M) similarities of each calculated image is performed by taking the arithmetic mean of the (N+M) similarities.

[0015] In one embodiment, the method further includes: after obtaining the optimal ranking of the combined search results, extracting the top K images in the optimal ranking as the optimal search results.

[0016] In one embodiment, the similarity is cosine similarity.

[0017] In a second aspect, embodiments of the present invention provide a combined search result optimization device, the combined search result optimization device comprising:

[0018] The combined search result input module is used to obtain combined search results for N face images to be searched; the combined search results include N image sequences, each image sequence is K images obtained from the search of the corresponding single face image to be searched, arranged in descending order of similarity, where N and K are both integers greater than zero;

[0019] The high-confidence search support set construction module is used to select M images from the combined search results that belong to the same person as the N face images to be retrieved to form a high-confidence search support set, and the remaining images form the remaining image set, where 1≤M<N;

[0020] The similarity update module is used to update the similarity of N×K images in the combined search results using the high-confidence search support set. For images in the high-confidence search support set, the overall similarity of each image relative to N images to be retrieved is calculated based on the similarity of each image with a single image of the face to be retrieved, and this is used as the updated similarity. For images in the remaining image set, the overall similarity of each image relative to N images of the face to be retrieved and M images in the high-confidence search support set is calculated based on the similarity of each image with a single image of the face to be retrieved and a single image in the high-confidence search support set, and this is used as the updated similarity.

[0021] The sorting module is used to sort the N×K images in the combined search results in descending order according to the updated similarity, so as to obtain the optimal sorting of the combined search results.

[0022] Thirdly, embodiments of the present invention provide a computer device, the computer device including a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the combined search result optimization method as described in the first aspect.

[0023] Fourthly, embodiments of the present invention provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the combined search result optimization method as described in the first aspect.

[0024] The beneficial effects of the embodiments of the present invention compared with the prior art are as follows:

[0025] Compared with existing technologies, this invention can greatly improve the sorting effect of search results for multiple face images, so that images belonging to the same person as the face image to be searched are placed as high as possible, and images of different people are placed as low as possible.

[0026] Specifically, before optimizing and ranking the combined search results of multiple face images, this invention first constructs a high-confidence search support set, ensuring that the images in the high-confidence search support set belong to the same person as the face image to be retrieved. Then, based on the high-confidence search support set, the similarity of each image in the combined search results is updated. The similarity of each image in the high-confidence search support set is updated to a comprehensive similarity relative to N face images to be retrieved, and the similarity of each image in the remaining image set is updated to a comprehensive similarity relative to the N face images to be retrieved and the M images in the high-confidence search support set. This optimizes and ranks the original search results. The similarity based on a single image is replaced by a comprehensive similarity based on multiple images. Since the comprehensive similarity based on multiple images fully considers the differences in facial features between different images, it can ensure that the higher the comprehensive similarity, the greater the probability that the image belongs to the same person as the image being searched. In other words, the comprehensive similarity based on multiple images can effectively determine whether an image is of the person in question. Consequently, when the combined search results are sorted in descending order according to the updated similarity, images belonging to the same person as the image being searched are placed as high as possible, while images of different people are placed as low as possible. Furthermore, when updating the similarity of each image in the remaining image set, this invention considers that the images in the high-confidence search support set belong to the same person as the face image to be retrieved. The images in the high-confidence search support set are used as a supplement to the face image to be retrieved. The comprehensive similarity of each image in the remaining image set relative to N face images to be retrieved and M high-confidence search support set images is calculated. The comprehensive similarity calculated in this way takes into account the representational differences between (N+M) images. The obtained comprehensive similarity can better reflect whether it is a picture of the person and can further improve the ranking effect of combined search results. Attached Figure Description

[0027] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0028] Figure 1 This is a flowchart illustrating a combined search result optimization method provided in Embodiment 1 of the present invention;

[0029] Figure 2 This is a schematic diagram of a high-confidence search support set construction method provided in Embodiment 1 of the present invention;

[0030] Figure 3 This is a schematic diagram of the similarity update of the remaining image set provided in Embodiment 1 of the present invention;

[0031] Figure 4 This is a flowchart illustrating a combined search result optimization method provided in Embodiment 2 of the present invention;

[0032] Figure 5 This is a schematic diagram of a high-confidence search support set construction method provided in Embodiment 2 of the present invention;

[0033] Figure 6 This is a schematic diagram of a combined search result optimization device provided in Embodiment 3 of the present invention;

[0034] Figure 7 This is a schematic diagram of the structure of a computer device provided in Embodiment 4 of the present invention. Detailed Implementation

[0035] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of the invention. However, those skilled in the art will understand that the invention can be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods are omitted so as not to obscure the description of the invention with unnecessary detail.

[0036] It should be understood that, when used in this specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0037] It should also be understood that the term "and / or" as used in this specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0038] As used in this specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if [described condition or event] is detected" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once [described condition or event] is detected," or "in response to detection of [described condition or event]."

[0039] Furthermore, in the description of this invention and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0040] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of the invention include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0041] It should be understood that the sequence number of each step in the following embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0042] The principles and spirit of the present invention will be explained in detail below with reference to several representative embodiments.

[0043] In existing technologies, to obtain more complete search results, multiple facial images of the same person are typically searched separately, and the union of the search results for each image is used as the combined search result. After obtaining the combined search results based on multiple facial images, the combined search results need to be sorted. The current mainstream approach is to directly deduplicate and sort, but this approach ignores the representational differences between images. Essentially, it is still based on the similarity calculated for individual images for deduplication and sorting, which cannot perfectly distinguish between images of the person and those of the person. There may be situations where high similarity does not necessarily mean the person is not represented, and low similarity does not necessarily mean the person is not represented. It fails to ensure that images of the person are placed as high as possible and images of non-persons are placed as low as possible.

[0044] To address the above issues, this invention proposes a combined search result optimization method. This method optimizes and sorts combined search results from multiple face images. The core of this method is to construct a "high-confidence search support set," and then update the similarity of each image in the combined search results based on the high-confidence search support set. The original similarity based on a single image is replaced by a comprehensive similarity based on multiple images. Since the comprehensive similarity based on multiple images fully considers the differences in facial feature representation between different images, it ensures that images with higher comprehensive similarity have a greater probability of belonging to the same person as the face image being searched. Finally, the combined search results are arranged in descending order of comprehensive similarity, thus ensuring that images belonging to the same person as the face image being searched are placed as high as possible, while images of different people are placed as low as possible.

[0045] After introducing the basic principles of the present invention, various non-limiting embodiments of the present invention will be described in detail below.

[0046] The present invention provides a combined search result optimization method for optimizing and sorting combined search results of multiple face images.

[0047] See Figure 1 This is a flowchart illustrating a combined search result optimization method provided in Embodiment 1 of the present invention, as shown below. Figure 1 As shown, this combined search result optimization method may include the following steps:

[0048] Step S101: Obtain the combined search results of N face images to be searched; wherein, the combined search results include N image sequences, each image sequence is K images obtained from the search of the corresponding single face image to be searched, arranged in descending order of similarity, and N and K are both integers greater than zero.

[0049] Specifically, the combined search results for N face images to be retrieved can be obtained as follows: First, each of the N face images to be retrieved is searched based on the face search database. After the search, each face image will yield an image sequence (each image sequence contains K images, and the K images are arranged in descending order of similarity). In fact, for a certain face image to be retrieved, these K images are the top K images most similar to that face image found in the face search database. Then, the union of the search results for the N face images to be retrieved is taken to obtain the combined search results. Therefore, the combined search results contain N image sequences, totaling N×K images.

[0050] Optionally, after taking the union of the search results of N face images to be searched to obtain the combined search results, there may be duplicate images among the N×K images obtained. In order to improve the processing effect of subsequent steps, after obtaining N×K images, a deduplication operation can be performed first, and then subsequent steps S102 to S104 can be performed based on the deduplicated combined search results.

[0051] Step S102: Divide the combined search results into a high-confidence search support set and a remaining image set;

[0052] Specifically, from the combined search results, M images belonging to the same person as the N facial images to be retrieved are selected to form a high-confidence search support set. The remaining images in the combined search results other than the high-confidence search support set are formed into the remaining image set, where 1 ≤ M < N.

[0053] In this embodiment, the method for constructing the high-confidence search support set is described in [link to documentation]. Figure 2 The details are as follows:

[0054] First, for the N image sequences in the combined search results, each image sequence is truncated using a preset similarity threshold to obtain N truncated sequences. In each truncated sequence, only images with a similarity greater than the preset similarity threshold are retained.

[0055] Then, the intersection of the N truncated sequences is taken, and the images that appear simultaneously in at least two truncated sequences are taken as high-confidence images. All the high-confidence images are used to form a high-confidence search support set.

[0056] Optionally, the "preset similarity threshold" can be set based on existing facial feature algorithms or empirical data, as long as it ensures that images with a similarity threshold greater than the threshold are likely to belong to the same person as the image being searched. For example, the preset similarity threshold can be set to 90% based on empirical data. In this case, if the similarity of an image is greater than 90%, it can be considered that the image is likely to belong to the same person as the image being searched.

[0057] In summary, this embodiment constructs a high-confidence search support set by setting a similarity threshold and taking the intersection. The purpose of taking the intersection is to ensure that only images that appear simultaneously in two or more truncated sequences are used to constitute the high-confidence search support set. The beneficial effect of doing so is to further improve the credibility of the high-confidence search support set, so that the images in the high-confidence search support set are as close as possible to the person's own image (i.e., the images in the high-confidence search support set and the image of the face to be retrieved are as close as possible to the same person), and to remove a small number of images that, although highly similar, are not of the person in question.

[0058] Optionally, to further improve the confidence of the high-confidence search support set, images that appear simultaneously in at least three truncated sequences can be used as high-confidence images, and these high-confidence images can then be used to construct the high-confidence search support set. It's easy to understand that the number 3 here can be replaced with other numbers as needed, such as L, where 2 ≤ L ≤ N.

[0059] Step S103: Update the similarity of N×K images in the combined search results using the high-confidence search support set;

[0060] Specifically, for images in the high-confidence search support set, the overall similarity of each image relative to N images in the search set is calculated based on the similarity between each image and a single image of the face to be retrieved, and this is used as the updated similarity. For images in the remaining image set, the overall similarity of each image relative to N images of the face to be retrieved and M images in the high-confidence search support set is calculated based on the similarity between each image and a single image of the face to be retrieved and a single image in the high-confidence search support set, and this is used as the updated similarity.

[0061] In this embodiment, for images in the high-confidence search support set, the similarity between each image and N images of the face to be searched is calculated. Then, the arithmetic mean of the N similarity scores for each image is taken to obtain the comprehensive similarity of each image relative to the N images of the face to be searched. The calculation formula is as follows:

[0062]

[0063] In the formula, x i This indicates that high-confidence search supports centralized sorting of the image at position i, s i Image x i Compared to the overall similarity of N face images to be retrieved, x j Let cosine(x) be the j-th image among N images to be searched. i ,x j ) represents image x i And image x j Cosine similarity between them.

[0064] For the remaining images in the image set, calculate the similarity between each image and the N images of the target face and the M images in the high-confidence search support set. Then, take the arithmetic mean of the (N+M) similarity scores for each image to obtain the comprehensive similarity of each image relative to the N images of the target face and the M images in the high-confidence search support set. The calculation formula is as follows:

[0065]

[0066] In the formula, x i Let s represent the image at position i in the remaining image set. i Image x i Compared to the combined similarity of N face images to be retrieved and M high-confidence search support set images, x j Let j be the j-th image in (N face images to be searched + M high-confidence search support set images), and let cosine(x) be the j-th image. i ,x j ) represents image x i And image x j Cosine similarity between them.

[0067] For example, such as Figure 3 As shown, assuming there are N = 2 face images to be searched, a search is performed in the face search database. For each face image to be searched, K = 4 images are found, resulting in a total of N × K = 8 combined search results. Among them, the high-confidence search support set contains M = 2 images, and the remaining image set contains N × KM = 6 images.

[0068] To calculate image 2 in the remaining image set (i.e.Figure 3 Taking the overall similarity of search result 2) as an example, the calculation process for the overall similarity of this image is as follows:

[0069] Step 1: Calculate the similarity between image 2 in the remaining image set and (N face images to be retrieved + M high-confidence search support set images), denoted as cosine(x2, x...). j ), j = 1, 2, 3, 4;

[0070] In the formula, x2 represents image 2 in the remaining image set, x j The j-th image represents (N face images to be searched + M high-confidence search support set images), because Figure 3 Since N = 2 and M = 2, that is, N + M = 4, the value of j ranges from 1 to 4.

[0071] Step 2: Calculate the comprehensive similarity of image 2 in the remaining image set with respect to (N face images to be retrieved + M high-confidence search support set images), denoted as s2:

[0072]

[0073] Since N+M=4, Step 1 calculates four similarities for image 2 in the remaining image set. Therefore, the arithmetic mean of these four similarities is needed as the comprehensive similarity of image 2 in the remaining image set relative to (N face images to be retrieved + M high-confidence search support set images).

[0074] In this embodiment, for images in the high-confidence search support set, the arithmetic mean of the N similarities calculated for each image is taken to obtain the comprehensive similarity of each image relative to the N images of the target face. For images in the remaining image set, the arithmetic mean of the (N+M) similarities calculated for each image is taken to obtain the comprehensive similarity of each image relative to the N images of the target face and the M images in the high-confidence search support set. This method of calculating the comprehensive similarity through statistical analysis improves the rationality of the comprehensive similarity calculation, thereby making the ranking results more reliable when sorting the combined search results according to the comprehensive similarity.

[0075] Optionally, when calculating the overall similarity, in addition to taking the arithmetic mean, other statistical analysis methods can be used, such as taking the difference of squares or the mean square deviation.

[0076] In this embodiment, cosine distance is used to measure the similarity of facial features (i.e., cosine similarity). In other implementations, Euclidean distance can also be used to measure the similarity of facial features.

[0077] Step S104: Sort the N×K images in the combined search results in descending order of their updated similarity to obtain the optimal ranking of the combined search results.

[0078] After the processing in step S103 above, the similarity of each image in the high-confidence search support set has been updated to a comprehensive similarity relative to N images of the face to be retrieved. The similarity of each image in the remaining image set has been updated to a comprehensive similarity relative to N images of the face to be retrieved and M images in the high-confidence search support set. This replaces the original single-image-based similarity with a comprehensive similarity based on multiple images. Since the comprehensive similarity based on multiple images fully considers the differences in facial features between different images, it can ensure that the higher the comprehensive similarity, the greater the probability that the image belongs to the same person as the image to be retrieved. In other words, the comprehensive similarity based on multiple images can effectively determine whether the image is of the person in question. As a result, when the combined search results are finally sorted in descending order according to the updated similarity, images belonging to the same person as the image to be retrieved are placed as high as possible, while images of different people are placed as low as possible.

[0079] Furthermore, in this embodiment, when updating the similarity of each image in the remaining image set, considering that the images in the high-confidence search support set belong to the same person as the face image to be retrieved, the images in the high-confidence search support set are used as a supplement to the face image to be retrieved. The comprehensive similarity of each image in the remaining image set relative to the N face images to be retrieved and the M high-confidence search support set images is calculated. In this way, the comprehensive similarity calculated simultaneously considers the representational differences between (N+M) images, and the obtained comprehensive similarity can better reflect whether it is the person's image, which can further improve the ranking effect of the combined search results.

[0080] See Figure 4 This is a flowchart illustrating a combined search result optimization method provided in Embodiment 2 of the present invention, as shown below. Figure 4 As shown, this combined search result optimization method may include the following steps:

[0081] Step S401: Obtain the combined search results of N face images to be searched; wherein, the combined search results include N image sequences, each image sequence is K images obtained from the search of the corresponding single face image to be searched, arranged in descending order of similarity, and N and K are both integers greater than zero.

[0082] Specifically, the combined search results for N face images to be retrieved can be obtained as follows: First, each of the N face images to be retrieved is searched based on the face search database. After the search, each face image will yield an image sequence (each image sequence contains K images, and the K images are arranged in descending order of similarity). In fact, for a certain face image to be retrieved, these K images are the top K images most similar to that face image found in the face search database. Then, the union of the search results for the N face images to be retrieved is taken to obtain the combined search results. Therefore, the combined search results contain N image sequences, totaling N×K images.

[0083] Optionally, after taking the union of the search results of N face images to be searched to obtain the combined search results, there may be duplicate images among the N×K images obtained. In order to improve the processing effect of subsequent steps, after obtaining N×K images, a deduplication operation can be performed first, and then subsequent steps S402 to S405 can be performed based on the deduplicated combined search results.

[0084] Step S402: Divide the combined search results into a high-confidence search support set and a remaining image set;

[0085] Specifically, from the combined search results, M images belonging to the same person as the N facial images to be retrieved are selected to form a high-confidence search support set. The remaining images in the combined search results other than the high-confidence search support set are formed into the remaining image set, where 1 ≤ M < N.

[0086] In this embodiment, the method for constructing the high-confidence search support set is described in [link to documentation]. Figure 5 The details are as follows:

[0087] First, for the N image sequences in the combined search results, each image sequence is truncated using a preset similarity threshold to obtain N truncated sequences. In each truncated sequence, only images with a similarity greater than the preset similarity threshold are retained.

[0088] Then, the union of the N truncated sequences is used to obtain the high-confidence search support set.

[0089] Optionally, the "preset similarity threshold" can be set based on existing facial feature algorithms or empirical data, as long as it ensures that images with a similarity threshold greater than the threshold are likely to belong to the same person as the image being searched. For example, the preset similarity threshold can be set to 90% based on empirical data. In this case, if the similarity of an image is greater than 90%, it can be considered that the image is likely to belong to the same person as the image being searched.

[0090] In summary, this embodiment constructs a high-confidence search support set by setting a similarity threshold and taking the union of N truncated sequences. By reasonably setting the similarity threshold, it can be ensured that the truncated sequences contain as many images of the person as possible. By taking the union of N truncated sequences, the high-confidence search support set can contain as many images of the person as possible.

[0091] Step S403: Update the similarity of N×K images in the combined search results using the high confidence search support set;

[0092] Specifically, for images in the high-confidence search support set, the overall similarity of each image relative to N images in the search set is calculated based on the similarity between each image and a single image of the face to be retrieved, and this is used as the updated similarity. For images in the remaining image set, the overall similarity of each image relative to N images of the face to be retrieved and M images in the high-confidence search support set is calculated based on the similarity between each image and a single image of the face to be retrieved and a single image in the high-confidence search support set, and this is used as the updated similarity.

[0093] In this embodiment, for images in the high-confidence search support set, the similarity between each image and N face images to be searched is calculated, and then the N similarities of each image are statistically analyzed to obtain the comprehensive similarity of each image relative to the N face images to be searched.

[0094] For the remaining images in the image set, calculate the similarity between each image and the N images of the face to be searched and the M images of the high-confidence search support set. Then, perform statistical analysis on the (N+M) similarity scores of each image to obtain the comprehensive similarity of each image relative to the N images of the face to be searched and the M images of the high-confidence search support set.

[0095] Optionally, when calculating the overall similarity, the statistical analysis methods that can be used include, but are not limited to, taking the arithmetic mean, taking the difference of squares, or taking the mean square deviation.

[0096] Step S404: Sort the N×K images in the combined search results in descending order of the updated similarity to obtain the optimal ranking of the combined search results;

[0097] After the processing in step S403 above, the similarity of each image in the high-confidence search support set has been updated to a comprehensive similarity relative to N images of the face to be retrieved. The similarity of each image in the remaining image set has been updated to a comprehensive similarity relative to N images of the face to be retrieved and M images in the high-confidence search support set. This replaces the original single-image-based similarity with a comprehensive similarity based on multiple images. Since the comprehensive similarity based on multiple images fully considers the differences in facial features between different images, it can ensure that the higher the comprehensive similarity, the greater the probability that the image belongs to the same person as the image to be retrieved. In other words, the comprehensive similarity based on multiple images can effectively determine whether the image is of the person in question. As a result, when the combined search results are finally sorted in descending order according to the updated similarity, images belonging to the same person as the image to be retrieved are placed as high as possible, while images of different people are placed as low as possible.

[0098] Furthermore, in this embodiment, when updating the similarity of each image in the remaining image set, considering that the images in the high-confidence search support set belong to the same person as the face image to be retrieved, the images in the high-confidence search support set are used as a supplement to the face image to be retrieved. The comprehensive similarity of each image in the remaining image set relative to the N face images to be retrieved and the M high-confidence search support set images is calculated. In this way, the comprehensive similarity calculated simultaneously considers the representational differences between (N+M) images, and the obtained comprehensive similarity can better reflect whether it is the person's image, which can further improve the ranking effect of the combined search results.

[0099] Step S405: After obtaining the optimal ranking of the combined search results, extract the top K images from the optimal ranking as the optimal search results.

[0100] Since the optimal sorting prioritizes images belonging to the same person as the face image being searched, and prioritizes images of different people, extracting the top K images from the optimal sorting as the best search result for multiple face images ensures that the extracted K images are all of the person in question, resulting in higher accuracy.

[0101] It should be noted that this embodiment adopts... Figure 5 The method shown constructs a high-confidence search support set, which ensures that the high-confidence search support set contains as many images of the individual as possible; as another implementation method, it can also be... Figure 2 The method shown constructs a high-confidence search support set, which can further improve the credibility of the high-confidence search support set, ensuring that the images in the high-confidence search support set are as similar as possible to the person, and removing a small number of images that, although highly similar, are not of the person.

[0102] Corresponding to the combined search result optimization method in the above embodiments, Figure 6The diagram shows a structural block diagram of a combined search result optimization device provided in Embodiment 3 of the present invention. The combined search result optimization device is used to optimize and sort combined search results of multiple face images.

[0103] See Figure 6 The combined search result optimization device includes:

[0104] The combined search results input module is used to obtain combined search results for N face images to be searched; wherein, the combined search results include N image sequences, each image sequence is K images obtained from the search of the corresponding single face image to be searched, arranged in descending order of similarity, and N and K are both integers greater than zero;

[0105] The high confidence search support set construction module is used to filter out M images from the combined search results that belong to the same person as N face images to be retrieved to form a high confidence search support set, and the remaining images form the remaining image set, 1≤M<N.

[0106] The similarity update module is used to update the similarity of N×K images in the combined search results using the high-confidence search support set. For images in the high-confidence search support set, the overall similarity of each image relative to the N images to be retrieved is calculated based on the similarity of each image with a single image of the face to be retrieved, and this is used as the updated similarity. For images in the remaining image set, the overall similarity of each image relative to the N images of the face to be retrieved and the M images in the high-confidence search support set is calculated based on the similarity of each image with a single image of the face to be retrieved and a single image in the high-confidence search support set, and this is used as the updated similarity.

[0107] The sorting module is used to sort the N×K images in the combined search results in descending order according to the updated similarity, so as to obtain the optimal sorting of the combined search results.

[0108] Optionally, the combined search result optimization device further includes an optimal search result output module, which is used to extract the top K images in the optimal ranking as the optimal search results after obtaining the optimal ranking of the combined search results.

[0109] Optionally, the high-confidence search support set building block also includes:

[0110] The image sequence truncation subunit is used to truncate each of the N image sequences in the combined search results using a preset similarity threshold to obtain N truncated sequences. Each truncated sequence retains only the images with a similarity greater than the preset similarity threshold.

[0111] The intersection set subunit is used to find the intersection of N truncated sequences. Images that appear simultaneously in at least two truncated sequences are considered high-confidence images. All high-confidence images are used to construct a high-confidence search support set.

[0112] Optionally, the high-confidence search support set building block also includes:

[0113] The image sequence truncation subunit is used to truncate each of the N image sequences in the combined search results using a preset similarity threshold to obtain N truncated sequences. In each truncated sequence, only images with a similarity greater than the similarity threshold are retained.

[0114] The union subunit is used to obtain a high-confidence search support set by taking the union of N truncated sequences.

[0115] Optionally, the similarity update module also includes:

[0116] The high-confidence search support unit includes an image similarity update subunit. This subunit calculates the similarity between each image in the high-confidence search support set and N images of faces to be searched. Then, it takes the arithmetic mean of the N similarities for each image to obtain the overall similarity between each image and the N images of faces to be searched. The calculation formula is as follows:

[0117]

[0118] In the formula, x i This indicates that high-confidence search supports centralized sorting of the image at position i, s i Image x i Compared to the overall similarity of N face images to be retrieved, x j Let cosine(x) be the j-th image among N images to be searched. i ,x j ) represents image x i And image x j Cosine similarity between them.

[0119] The image similarity update subunit in the remaining image set is used to calculate the similarity between each image in the remaining image set and N images of the target face and M images in the high-confidence search support set. Then, the arithmetic mean of the (N+M) similarity scores for each image is taken to obtain the comprehensive similarity of each image relative to the N images of the target face and the M images in the high-confidence search support set. The calculation formula is as follows:

[0120]

[0121] In the formula, x i Let s represent the image at position i in the remaining image set. i Image x i Compared to the combined similarity of N face images to be retrieved and M high-confidence search support set images, x jLet j be the j-th image in (N face images to be searched + M high-confidence search support set images), and let cosine(x) be the j-th image. i ,x j ) represents image x i And image x j Cosine similarity between them.

[0122] It should be noted that the information interaction and execution process between the above modules, units, and sub-units are based on the same concept as the method embodiments of the present invention. For details on their specific functions and technical effects, please refer to the method embodiments section, which will not be repeated here.

[0123] Figure 7 This is a schematic diagram of the structure of a computer device provided in Embodiment 4 of the present invention. Figure 7 As shown, the computer device of this embodiment includes: at least one processor ( Figure 7 Only one is shown in the diagram), a memory, and a computer program stored in the memory and executable on at least one processor, wherein the processor executes the computer program to implement the steps in any of the above-described relation identification method embodiments.

[0124] This computer device may include, but is not limited to, a processor and memory. Those skilled in the art will understand that... Figure 7 The examples of computer devices are merely examples and do not constitute a limitation on computer devices. Computer devices may include more or fewer components than shown in the illustration, or combinations of certain components, or different components, such as network interfaces, displays, and input devices.

[0125] The processor referred to can be a CPU, but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0126] Memory includes readable storage media, internal memory, etc., wherein internal memory can be the RAM of a computer device, providing an environment for the operation of the operating system and computer-readable instructions stored in the readable storage media. The readable storage media can be the hard drive of a computer device, or in other embodiments, it can be an external storage device of the computer device, such as a plug-in hard drive, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card. Furthermore, memory can include both internal storage units and external storage devices of a computer device. Memory is used to store the operating system, applications, bootloader, data, and other programs, such as program code for computer programs. Memory can also be used to temporarily store data that has been output or will be output.

[0127] Those skilled in the art will understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the functions described above can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this invention. The specific working process of the units and modules in the above device can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here. If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention can implement all or part of the processes in the methods of the above embodiments by instructing related hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the above method embodiments. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. A computer-readable medium can include at least: any entity or device capable of carrying computer program code, a recording medium, a computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0128] The present invention can implement all or part of the processes in the methods of the above embodiments, or it can be accomplished by a computer program product. When the computer program product is run on a computer device, the computer device executes the steps in the above method embodiments.

[0129] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0130] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0131] In the embodiments provided by this invention, it should be understood that the disclosed apparatus / computer devices and methods can be implemented in other ways. For example, the apparatus / computer device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0132] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0133] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A method for optimizing a combination search result, the method comprising: The combined search result optimization method comprises: obtaining a combined search result of N face pictures to be searched; the combined search result comprises N picture sequences, each picture sequence being K pictures searched according to a corresponding single face picture to be searched and arranged in descending order of similarity, N and K being integers greater than zero; screening M pictures belonging to the same person as the N face pictures to be searched from the combined search result to form a high-confidence search support set, and the rest of the pictures forming a remaining picture set, 1≤M<N; updating the similarity of N×K pictures in the combined search result by using the high-confidence search support set; for the pictures in the high-confidence search support set, calculating a comprehensive similarity of each picture relative to the N face pictures to be searched based on the similarity of each picture to a single face picture to be searched as the updated similarity; for the pictures in the remaining picture set, calculating a comprehensive similarity of each picture relative to the N face pictures to be searched and the M pictures in the high-confidence search support set based on the similarity of each picture to a single face picture to be searched and a single picture in the high-confidence search support set as the updated similarity; arranging the N×K pictures in the combined search result in descending order of the updated similarity to obtain an optimal order of the combined search result.

2. The combined search result optimization method of claim 1, wherein, Screening M pictures belonging to the same person as the N face pictures to be searched from the combined search result to form a high-confidence search support set comprises: for the N picture sequences in the combined search result, truncating each picture sequence by using a preset similarity threshold to obtain N truncated sequences, each truncated sequence retaining only pictures with a similarity greater than the similarity threshold, taking the intersection of the N truncated sequences, taking pictures appearing in at least L truncated sequences as high-confidence pictures, and using all the high-confidence pictures to form the high-confidence search support set, 2≤L≤N.

3. The combined search result optimization method of claim 1, wherein, Screening M pictures belonging to the same person as the N face pictures to be searched from the combined search result to form a high-confidence search support set comprises: for the N picture sequences in the combined search result, truncating each picture sequence by using a preset similarity threshold to obtain N truncated sequences, each truncated sequence retaining only pictures with a similarity greater than the similarity threshold, and taking the union of the N truncated sequences to obtain the high-confidence search support set.

4. The combined search result optimization method of any of claims 1-3, wherein, For the pictures in the high-confidence search support set, calculating a comprehensive similarity of each picture relative to the N face pictures to be searched based on the similarity of each picture to a single face picture to be searched comprises: calculating the similarity of each picture to the N face pictures to be searched, respectively, and then statistically analyzing and processing the N similarities of each picture to obtain the comprehensive similarity of each picture relative to the N face pictures to be searched. For the pictures in the remaining picture set, based on the similarity of each picture to a single picture to be searched and a single picture in the high-confidence search support set, the comprehensive similarity of each picture to the N pictures to be searched and the M pictures in the high-confidence search support set is calculated, including: for the pictures in the remaining picture set, the similarity of each picture to the N pictures to be searched and the M pictures in the high-confidence search support set is calculated respectively, and then statistical analysis is performed on the (N+M) similarities of each picture to obtain the comprehensive similarity of each picture to the N pictures to be searched and the M pictures in the high-confidence search support set.

5. The combined search result optimization method of claim 4, wherein, The statistical analysis of the N similarities of each picture is to take the arithmetic mean of the N similarities; and the statistical analysis of the (N+M) similarities of each picture is to take the arithmetic mean of the (N+M) similarities.

6. The combined search result optimization method of claim 5, wherein, The method further includes: after obtaining the optimal ranking of the combined search result, extracting the first K pictures in the optimal ranking as the optimal search result.

7. The combined search result optimization method of claim 6, wherein, The similarity is a cosine similarity.

8. A combination search result optimization apparatus, comprising: The combined search result optimization apparatus includes: A combined search result input module configured to obtain a combined search result of N pictures to be searched; the combined search result includes N picture sequences, each picture sequence is K pictures searched for a corresponding single picture to be searched and arranged in descending order of similarity, and N and K are both integers greater than zero; A high-confidence search support set construction module configured to filter M pictures belonging to the same person as the N pictures to be searched from the combined search result to form a high-confidence search support set, and the remaining pictures form a remaining picture set, and 1≤M A similarity updating module configured to update the similarity of N×K pictures in the combined search result using the high-confidence search support set; for the pictures in the high-confidence search support set, based on the similarity of each picture to a single picture to be searched, the comprehensive similarity of each picture to the N pictures to be searched is calculated as the updated similarity; for the pictures in the remaining picture set, based on the similarity of each picture to a single picture to be searched and a single picture in the high-confidence search support set, the comprehensive similarity of each picture to the N pictures to be searched and the M pictures in the high-confidence search support set is calculated as the updated similarity; A ranking module configured to arrange the N×K pictures in the combined search result in descending order of the updated similarity to obtain an optimal ranking of the combined search result.

9. A computer device, comprising: The computer device includes a processor, a memory, and a computer program stored in the memory and executable on the processor, and the processor implements the combined search result optimization method of any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium storing a computer program, the computer program comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 9. The computer program is executed by the processor to implement the combined search result optimization method of any one of claims 1 to 7.

Images