A sam-based intelligent image search method

Abstract

The application provides a kind of intelligent image search method based on SAM, it is related to image processing technical field.The method comprises: inputting the image to be searched into SAM segmentation model, obtains the segmentation mask of all objects in the image to be searched;According to the object, select the object to be searched, obtain the segmentation mask corresponding to the search object image;According to the image to be searched and the segmentation mask corresponding to the search object image, determine the target object image;Obtain the feature vector of the target object image;Create image vector warehouse, obtain the feature vector of each image of the image vector warehouse;Determine the cosine distance between each image of the image vector warehouse and the target object image;According to the cosine distance, determine the most similar image of the target object image in the image vector warehouse.According to the application, the detail features of the search object can be highlighted, and part of the features of the surrounding object can be retained, and the search accuracy can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and in particular to an intelligent image search method based on SAM. BACKGROUND

[0002] In the related art, CN116204671A provides an image search method and device, relating to the technical field of artificial intelligence, specifically to the technical field of computer vision, image processing, deep learning, etc. According to the disclosed image search method, it includes: performing image search on a to-be-searched image to obtain a first search result from a first database; extracting text information from the to-be-searched image; performing text search based on the extracted text information to obtain a second search result from a second database; and outputting a search result for the to-be-searched image based on the first search result and the second search result.

[0003] CN115248871A provides an image search method and device, the method includes: obtaining a decoupling feature vector of a to-be-searched face image; inputting the decoupling feature vector into a target classifier to obtain a feature category to which the to-be-searched face image belongs; determining a first middle layer feature vector of the to-be-searched face image according to the decoupling feature vector and the feature category to which the to-be-searched face image belongs; and matching the first middle layer feature vector with a second middle layer feature vector of each image in a to-be-searched image set to obtain a target face image corresponding to the to-be-searched face image. This application obtains the decoupling feature vector of the face image, constructs a series of middle layer features based on the decoupling feature vector, and uses it for face search, solving the problem of sharp decline in face search efficiency and search accuracy due to key point occlusion and inaccurate key point positioning when the amount of face data rises in the prior art, and improving the speed and accuracy of face search.

[0004] Therefore, in the related art, although the similarity between the overall feature vector of the query image and the vector in the database can be calculated to find the most similar image, the overall feature vector cannot accurately capture the details of the target object, and it is difficult to highlight the detailed features of the search object and retain part of the features of the surrounding objects when searching for a specific object in a complex background.

[0005] The information disclosed in the Background section of the present application is only intended to deepen the understanding of the general background of the present application, and should not be regarded as acknowledging or implying in any form that the information constitutes prior art known to those skilled in the art. SUMMARY

[0006] The present application provides an intelligent image search method based on SAM, which can solve the technical problem of being difficult to highlight the detailed features of the search object and retain part of the features of the surrounding objects.

[0007] According to the application, a SAM-based intelligent image search method is provided, which comprises: inputting a to-be-searched image into a SAM segmentation model to obtain a segmentation mask of all objects in the to-be-searched image, wherein the segmentation mask and the to-be-searched image are matrices of the same size, and the elements of the segmentation mask matrix take values of 0 or 1, 0 being a background and 1 being a target object; selecting an object to be searched according to the objects to obtain a segmentation mask corresponding to a search object image; determining a target object image according to the to-be-searched image and the segmentation mask corresponding to the search object image; obtaining a feature vector of the target object image according to the target object image; creating an image vector warehouse to obtain a feature vector of each image in the image vector warehouse; determining a cosine distance between each image in the image vector warehouse and the target object image according to the feature vector of the target object image and the feature vector of each image in the image vector warehouse; and determining an image in the image vector warehouse that is most similar to the target object image according to the cosine distance.

[0008] According to the application, the target object image is determined according to the to-be-searched image and the segmentation mask corresponding to the search object image, which comprises: selecting whether the search object image needs to consider surrounding object features; when the search object image needs to consider surrounding object features, performing blur processing on the to-be-searched image to determine a blurred image; determining an image in which an object part is clear and surrounding objects are blurred according to the blurred image, the to-be-searched image, and the segmentation mask corresponding to the search object image; determining the target object image as the image in which the object part is clear and the surrounding objects are blurred; when the search object image does not need to consider surrounding object features, determining an image in which an object part is clear and surrounding objects are black according to the to-be-searched image and the segmentation mask corresponding to the search object image; finding an image contour in which pixels of the image in which the object part is clear and the surrounding objects are black are not 0, extracting the image in which the pixels are not 0 to obtain an image containing only an object; and determining the target object image as the image containing only the object.

[0009] According to the application, when the search object image needs to consider surrounding objects, the to-be-searched image is subjected to blur processing to determine a blurred image, which comprises: obtaining horizontal and vertical coordinates of pixels of the to-be-searched image; setting a size of a Gaussian convolution kernel and obtaining horizontal and vertical coordinates of the Gaussian convolution kernel; and determining the blurred image according to the to-be-searched image, the horizontal and vertical coordinates of the pixels of the to-be-searched image, the size of the Gaussian convolution kernel, and the horizontal and vertical coordinates of the Gaussian convolution kernel.

[0010] According to the application, the blurred image is determined according to the to-be-searched image, the horizontal and vertical coordinates of the pixels of the to-be-searched image, the size of the Gaussian convolution kernel, and the horizontal and vertical coordinates of the Gaussian convolution kernel, which comprises: determining the blurred image according to a formula determining a blurred image I blur wherein x is the horizontal coordinate of the pixel to be searched in the image, y is the vertical coordinate of the pixel to be searched in the image, K is a Gaussian convolution kernel, w is the horizontal coordinate of the Gaussian convolution kernel, h is the vertical coordinate of the Gaussian convolution kernel, t is the size of the Gaussian convolution kernel, k is the value of the parity of t, I input is the image to be searched.

[0011] According to the application, according to the segmentation mask corresponding to the blurred image, the image to be searched and the search object image, the image in which the object part is clear and the surrounding object is blurred is determined, comprising: according to the formula I1 = mask·I input +(1-mask)·I blur , determining the image I1 in which the object part is clear and the surrounding object is blurred, wherein mask is the segmentation mask corresponding to the search object image, I input is the image to be searched, and I blur is the blurred image.

[0012] According to the application, when the search object image does not need to consider the characteristics of the surrounding object, according to the segmentation mask corresponding to the image to be searched and the search object image, the image in which the object part is clear and the surrounding object is black is determined, comprising: according to the formula I2 = mask·I input , determining the image I2 in which the object part is clear and the surrounding object is black, wherein mask is the segmentation mask corresponding to the search object image, I input is the image to be searched.

[0013] According to the application, according to the feature vector of the target object image and the feature vector of each image in the image vector repository, the cosine distance between each image in the image vector repository and the target object image is determined, comprising: obtaining the multi-dimensional feature values of the feature vector of the target object image; obtaining the multi-dimensional feature values of the feature vector of each image in the image vector repository, wherein the dimension of the feature vector of each image in the image vector repository is consistent with the dimension of the feature vector of the target object image; according to the multi-dimensional feature values of the feature vector of the target object image and the multi-dimensional feature values of the feature vector of each image in the image vector repository, the cosine distance between each image in the image vector repository and the target object image is determined.

[0014] According to the application, according to the multi-dimensional feature values of the feature vector of the target object image and the multi-dimensional feature values of the feature vector of each image in the image vector repository, the cosine distance between each image in the image vector repository and the target object image is determined, comprising: according to the formula determining the cosine distance d(v, V I), wherein v is a feature vector of the target object image, V I is a feature vector of the Ith image of the image vector warehouse, v i is an ith eigenvalue of the feature vector of the target object image, V i is an ith eigenvalue of the Ith image of the image vector warehouse, n is a dimension of the feature vector of the target object image and the feature vector of each image of the image vector warehouse, i≤n, and I, i and n are positive integers.

[0015] According to the present application, the cosine distance is used to determine the image in the image vector warehouse that is most similar to the target object image, which includes: sorting the cosine distance between each image of the image vector warehouse and the target object image from small to large; and the image ranked first is the image in the image vector warehouse that is most similar to the target object image.

[0016] Technical effects: According to the present application, by using the SAM segmentation model, the panoramic segmentation of the image to be searched can be performed, and the image of any object in the image to be searched can be selected and extracted, and the most similar image can be searched. For the image in which the search object is clear and the surrounding object is blurred, the details of the search object can be highlighted, and part of the features of the surrounding object can be retained, thereby improving the search accuracy and more effectively searching for specific objects in a complex background. When determining the blurred image, the image to be searched can be blurred based on the Gaussian convolution filter blur formula, thereby determining the blurred image, which can effectively reduce the image quality and weaken the interference of the surrounding object, thereby helping to retain part of the features of the surrounding object. When determining the image in which the object part is clear and the surrounding object is blurred, the image to be searched and the blurred image can be mixed through the segmentation mask, thereby determining the image in which the object part is clear and the surrounding object is blurred, and in which the search object part is clear and the surrounding object is blurred. The search object in the image can be more prominent, the interference of the surrounding object can be weakened, and the recognition degree of the search object can be improved. When determining the image in which the object part is clear and the surrounding object is black, the pixel values of the part corresponding to the search object in the image to be searched can be retained through the segmentation mask, and the other parts can be set to black, thereby determining the image in which the object part is clear and the surrounding object is black. The search object can be highlighted, which can be more prominent in the image, and the background interference can be reduced by setting the surrounding area to black. When determining the cosine distance between the Ith image of the image vector warehouse and the target object image, the cosine distance between the target object image and each image in the image vector warehouse can be determined based on the vector distance between the feature vector of the target object image and the feature vector of each image of the image vector warehouse, which helps to effectively compare the similarity in image search and recognition, thereby finding the most similar image in the image vector warehouse.

[0017] It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the application, as claimed. Other features and aspects of the present application will become apparent from a review of the following detailed description, taken in conjunction with the BRIEF DESCRIPTION OF DRAWINGS

[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the drawings required for use in the embodiments or the prior art description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other embodiments can be obtained from these drawings without creative labor;

[0019] Figure 1 An exemplary flowchart of a SAM-based intelligent image search method according to an embodiment of the present application is shown;

[0020] Figure 2 An exemplary image vector warehouse creation diagram according to an embodiment of the present application is shown;

[0021] Figure 3 An exemplary basic framework diagram of a SAM-based intelligent image search method according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0022] In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0023] The technical solutions of the present application will be described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and some embodiments may not be described again for the same or similar concepts or processes.

[0024] Figure 1An example of a flowchart of a SAM-based intelligent image search method according to an embodiment of the present application is shown in the figure. The method includes: step S101, inputting a to-be-searched image into a SAM segmentation model to obtain a segmentation mask of all objects in the to-be-searched image, wherein the segmentation mask and the to-be-searched image are matrices of the same size, and the elements of the segmentation mask matrix take values of 0 or 1, 0 being a background and 1 being a target object; step S102, selecting an object to be searched according to the objects to obtain a segmentation mask corresponding to a search object image; step S103, determining a target object image according to the to-be-searched image and the segmentation mask corresponding to the search object image; step S104, obtaining a feature vector of the target object image according to the target object image; step S105, creating an image vector warehouse to obtain a feature vector of each image in the image vector warehouse; step S106, determining a cosine distance between each image in the image vector warehouse and the target object image according to the feature vector of the target object image and the feature vector of each image in the image vector warehouse; and step S107, determining an image in the image vector warehouse that is most similar to the target object image according to the cosine distance.

[0025] The SAM-based intelligent image search method according to an embodiment of the present application can perform panoramic segmentation on a to-be-searched image by using a SAM segmentation model, select and extract an image of any object in the to-be-searched image, and search for the most similar image. For an image in which a search object is clear and surrounding objects are blurred, the details of the search object can be highlighted, part of the features of the surrounding objects can be retained, the search accuracy can be improved, and specific objects can be more effectively searched in a complex background.

[0026] According to an embodiment of the present application, in step S101, the SAM segmentation model has the capabilities of instance segmentation and semantic segmentation, wherein the instance segmentation can identify and divide each object instance in an image, and the semantic segmentation can label each pixel in the image with a corresponding class label. The SAM segmentation model segments all objects in the to-be-searched image and generates segmentation masks of all the objects, the segmentation masks and the to-be-searched image are matrices of the same size, and the elements of the segmentation mask matrix take values of 0 or 1, if the value of the segmentation mask matrix at a corresponding position is 0, the pixel at the corresponding position in the to-be-searched image is a background, and if the value of the segmentation mask matrix at the corresponding position is 1, the pixel at the corresponding position in the to-be-searched image is a target object. Through the segmentation mask matrix, the background part and the target object in the to-be-searched image can be clearly distinguished.

[0027] According to an embodiment of the present application, in step S102, an image of the selected object is extracted from the to-be-searched image to form a search object image, and a segmentation mask corresponding to the search object image is obtained.

[0028] According to one embodiment of the present application, in step S103, according to the image to be searched and the segmentation mask corresponding to the search object image, further object recognition is performed, which helps to accurately extract the target object image needed.

[0029] According to one embodiment of the present application, step S103 comprises: selecting whether the search object image needs to consider the surrounding object features; when the search object image needs to consider the surrounding object features, performing blur processing on the image to be searched to determine a blurred image; according to the blurred image, the image to be searched, and the segmentation mask corresponding to the search object image, determining an image in which the object part is clear and the surrounding object is blurred; determining the target object image as the image in which the object part is clear and the surrounding object is blurred; when the search object image does not need to consider the surrounding object features, according to the image to be searched and the segmentation mask corresponding to the search object image, determining an image in which the object part is clear and the surrounding is black; finding the image contour of the image in which the object part is clear and the surrounding is black, and then extracting the image in which the pixel is not 0 to obtain an image containing only the object; and determining the target object image as the image containing only the object.

[0030] According to one embodiment of the present application, whether the search object image needs to consider the surrounding object features is selected, that is, when a larger search object is processed, it is selected to consider the surrounding object features, so that the required picture can be correctly searched, and when a smaller object is processed or the surrounding object features do not need to be considered, only the features of the smaller object need to be focused on, and it is selected not to consider the surrounding object features. In the case of selecting to consider the surrounding object features, part of the features of the surrounding object is retained, but the features of the surrounding object cannot be too prominent, so that the surrounding object is blurred, the detailed features of the search object are highlighted, and part of the features of the surrounding object is retained. The image to be searched is blurred using a Gaussian convolution filter blur formula to obtain a blurred image. The clear object and the blurred surrounding object are merged using the segmentation mask corresponding to the search object image to obtain an image in which the object part is clear and the surrounding object is blurred. When the search object image is small or the surrounding object features do not need to be considered, the search object can be cut out as the target object image using the segmentation mask corresponding to the search object image, and all features of the surrounding object are removed. An object in a picture is irregular, so the search object is first cut out using the findContours function, and the picture is rectangular, so black pixels are supplemented around to make the target object image just rectangular.

[0031] According to one embodiment of the present application, when the search object image needs to consider the surrounding object, the to-be-searched image is blurred to determine a blurred image, including: obtaining the horizontal and vertical coordinates of the to-be-searched image pixels; setting the size of the Gaussian convolution kernel and obtaining the horizontal and vertical coordinates of the Gaussian convolution kernel; and determining the blurred image according to the to-be-searched image, the horizontal and vertical coordinates of the to-be-searched image pixels, the size of the Gaussian convolution kernel and the horizontal and vertical coordinates of the Gaussian convolution kernel.

[0032] According to one embodiment of the present application, the horizontal and vertical coordinates of each pixel in the to-be-searched image are obtained, which can accurately locate and process each pixel in subsequent processing. The Gaussian convolution kernel is used for image blurring, and the size of the Gaussian convolution kernel determines the intensity of the blurring effect.

[0033] According to one embodiment of the present application, the blurred image is determined according to the to-be-searched image, the horizontal and vertical coordinates of the to-be-searched image pixels, the size of the Gaussian convolution kernel and the horizontal and vertical coordinates of the Gaussian convolution kernel, including: determining the blurred image I blur ,

[0034]

[0035] wherein x is the horizontal coordinate of the to-be-searched image pixel, y is the vertical coordinate of the to-be-searched image pixel, K is the Gaussian convolution kernel, w is the horizontal coordinate of the Gaussian convolution kernel, h is the vertical coordinate of the Gaussian convolution kernel, t is the size of the Gaussian convolution kernel, k is the value of the parity of t, and I input is the to-be-searched image.

[0036] According to one embodiment of the present application, in formula (2), represents that if the size of the Gaussian convolution kernel is even, k takes the size of the Gaussian convolution kernel, and if the size of the Gaussian convolution kernel is odd, k takes an integer of the size of the Gaussian convolution kernel minus 1, for example, when the size of the Gaussian convolution kernel is 3, the size of the Gaussian convolution kernel is odd, and k takes 2, and when the size of the Gaussian convolution kernel is 4, the size of the Gaussian convolution kernel is even, and k takes 4. In formula (1), represents that all the pixels in the convolution window are weighted and summed, the pixel value is weighted by the Gaussian convolution kernel K, and the blurred pixel value, i.e., the blurred image, is obtained. The larger the Gaussian convolution kernel, the more obvious the image blurring effect, and vice versa, the smaller the Gaussian convolution kernel, the lighter the blurring effect, and more image details can be retained.

[0037] In this way, the to-be-searched image can be blurred based on the Gaussian convolution filter blurring formula to determine the blurred image, which can effectively reduce the image quality and weaken the interference of the surrounding object, thereby helping to retain part of the features of the surrounding object.

[0038] According to one embodiment of the present application, the image with the object part clear and the surrounding object blurred is determined according to the segmentation mask corresponding to the blurred image, the image to be searched and the search object image, comprising: determining the image I1 with the object part clear and the surrounding object blurred according to formula (3),

[0039] I1 = mask I input + (1-mask) I blur (3)

[0040] wherein, mask is the segmentation mask corresponding to the search object image, I input is the image to be searched, and I blur is the blurred image.

[0041] According to one embodiment of the present application, in formula (3), mask I input indicates that the pixels corresponding to the search object in the image to be searched are reserved, i.e., the object part is clear. blur (1-mask) I indicates that the pixels other than the part corresponding to the search object in the blurred image are blurred, i.e., the surrounding object is blurred. The sum of the above two is the image with the object part clear and the surrounding object blurred.

[0042] In this way, the image to be searched and the blurred image can be mixed through the segmentation mask to determine the image with the object part clear and the surrounding object blurred, in which the search object part is clear and the surrounding object is blurred, which can make the search object in the image more prominent, weaken the interference of the surrounding object, and help to improve the recognition of the search object.

[0043] According to one embodiment of the present application, when the search object image does not need to consider the feature of the surrounding object, the image with the object part clear and the surrounding black is determined according to the segmentation mask corresponding to the image to be searched and the search object image, comprising: determining the image I2 with the object part clear and the surrounding black according to formula (4),

[0044] I2 = mask I input (4)

[0045] wherein, mask is the segmentation mask corresponding to the search object image, and I input is the image to be searched.

[0046] According to one embodiment of the present application, in formula (4), mask I input indicates that the pixels corresponding to the search object in the image to be searched are reserved, i.e., the image with the object part clear and the surrounding black.

[0047] In this way, the pixel values of the part corresponding to the search object in the to-be-searched image can be reserved through the segmentation mask, and other parts are set to black, a clear object part is determined, and the surrounding black image is clear, the search object can be highlighted, the search object is more prominent in the image, and the surrounding area is set to black, so that background interference is reduced.

[0048] According to an embodiment of the present application, in step S104, the target object image is put into the Image Embedding model to obtain the feature vector of the target object image. For example, the Image Encoder of CLIP is used as the Image Embedding.

[0049] According to an embodiment of the present application, in step S105, Figure 2 An image vector warehouse creation diagram according to an embodiment of the present application is exemplarily shown. The purpose of creating the image vector warehouse is to effectively store and search images. By extracting the feature vector of the image and recording the path thereof, efficient image retrieval can be achieved. Each image in the image vector warehouse is put into the Image Embedding model (the same model as that used for the target object image), for example, the Image Embedding of CLIP can be used to extract the feature vector of the image. After the feature vector of the image is extracted, the feature vector is recorded, and a mapping relationship between the image path and the feature vector is established for subsequent retrieval and matching.

[0050] According to an embodiment of the present application, in step S106, the cosine distance can be used to calculate the distance between two vectors. By calculating the cosine distance, the similarity between each image in the image vector warehouse and the target object image can be obtained.

[0051] According to an embodiment of the present application, step S106 includes: obtaining a multi-dimensional feature value of the feature vector of the target object image; obtaining a multi-dimensional feature value of the feature vector of each image in the image vector warehouse, wherein the dimension of the feature vector of each image in the image vector warehouse is consistent with the dimension of the feature vector of the target object image; and determining the cosine distance between each image in the image vector warehouse and the target object image according to the multi-dimensional feature value of the feature vector of the target object image and the multi-dimensional feature value of the feature vector of each image in the image vector warehouse.

[0052] According to an embodiment of the present application, the cosine distance between each image in the image vector warehouse and the target object image is determined according to the multi-dimensional feature value of the feature vector of the target object image and the multi-dimensional feature value of the feature vector of each image in the image vector warehouse, including: determining the cosine distance d(v, V I ),

[0053]

[0054] wherein v is the feature vector of the target object image, V I is the feature vector of the Ith image in the image vector repository, v i is the ith eigenvalue of the feature vector of the target object image, V i is the ith eigenvalue of the Ith image in the image vector repository, n is the dimension of the feature vector of the target object image and the feature vector of each image in the image vector repository, i≤n, and I, i and n are positive integers.

[0055] According to an embodiment of the present application, in formula (5), is the cosine distance between the feature vector of the Ith image in the image vector repository and the feature vector of the target object image, i.e., the cosine distance between the Ith image in the image vector repository and the target object image. The smaller the cosine distance, the more similar the feature vectors of the two images. is the numerical calculation of the cosine distance, is the summation of the eigenvalues of the corresponding dimensions of the feature vector of the target object image and the feature vector of the Ith image in the image vector repository, is the product of the norms of the feature vector of the target object image and the feature vector of the Ith image in the image vector repository.

[0056] In this way, the cosine distance between the target object image and each image in the image vector repository can be determined based on the vector distance between the feature vector of the target object image and the feature vector of each image in the image vector repository, which helps to perform effective similarity comparison in image search and recognition, so as to find the most similar image in the image vector repository.

[0057] According to an embodiment of the present application, in step S107, the images in the image vector repository can be sorted or filtered by the cosine distance, so as to find the most similar image to the target object image in the image vector repository.

[0058] According to an embodiment of the present application, step S107 comprises: sorting the images in the image vector repository according to the cosine distance from small to large; and the image ranked first is the most similar image to the target object image in the image vector repository.

[0059] According to an embodiment of the present application, the image ranked first in the sorting list, i.e., the image with the smallest cosine distance, is the most similar image to the target object image in the image vector repository. The similarity between this image and the target object image is the highest, and it can be regarded as the most matched image.

[0060] The SAM-based intelligent image search method according to the embodiments of the present application can perform panoramic segmentation on the image to be searched by using the SAM segmentation model, and select an image extracting any object in the image to be searched, and can search the most similar image. For the image in which the search object is clear and the surrounding object is blurred, the details of the search object can be highlighted, and part of the features of the surrounding object can be retained, the search accuracy can be improved, and the specific object can be searched more effectively in the complex background. When determining the blurred image, the image to be searched can be blurred based on the Gaussian convolution filter blur formula, so as to determine the blurred image, which can effectively reduce the image quality and weaken the interference of the surrounding object, thereby helping to retain part of the features of the surrounding object. When determining the image in which the object part is clear and the surrounding object is blurred, the image to be searched and the blurred image can be mixed through the segmentation mask, so as to determine the image in which the object part is clear and the surrounding object is blurred, in which the search object part is clear and the surrounding object is blurred, the search object in the image can be highlighted, the interference of the surrounding object can be weakened, and the recognition of the search object can be improved. When determining the image in which the object part is clear and the surrounding object is black, the pixel values of the part corresponding to the search object in the image to be searched can be retained through the segmentation mask, and the other parts can be set to black, so as to determine the image in which the object part is clear and the surrounding object is black, which can highlight the search object and make it more prominent in the image, and at the same time set the surrounding area to black, thereby reducing the background interference. When determining the cosine distance between the Ith image in the image vector warehouse and the target object image, the cosine distance between the target object image and each image in the image vector warehouse can be determined based on the vector distance between the feature vector of the target object image and the feature vector of each image in the image vector warehouse, which helps to effectively compare the similarity in image search and recognition, so as to find the most similar image in the image vector warehouse.

[0061] Figure 3 The basic framework of the SAM-based intelligent image search method according to the embodiments of the present application is exemplarily shown. By using the SAM segmentation model, all objects in the input image can be segmented, that is, an image extracting any object in the image can be selected, and the most similar image can be searched.

[0062] It should be understood by those skilled in the art that the embodiments of the present application shown in the above description and the drawings are only examples and do not limit the present application. The purpose of the present application has been fully and effectively achieved. The function and structural principle of the present application has been shown and described in the embodiments, and the embodiments of the present application can have any variation or modification without departing from the principle.

Claims

1. A SAM-based intelligent image search method, characterized in that, The method comprises the following steps: inputting a to-be-searched image into a SAM segmentation model to obtain a segmentation mask of all objects in the to-be-searched image, wherein the segmentation mask and the to-be-searched image are matrices of the same size, and the elements of the segmentation mask matrix take values of 0 or 1, 0 being a background and 1 being a target object; selecting an object to be searched according to the objects to obtain a segmentation mask corresponding to a search object image; determining a target object image according to the to-be-searched image and the segmentation mask corresponding to the search object image; obtaining a feature vector of the target object image according to the target object image; creating an image vector warehouse to obtain a feature vector of each image in the image vector warehouse; determining a cosine distance between each image in the image vector warehouse and the target object image according to the feature vector of the target object image and the feature vector of each image in the image vector warehouse; determining an image in the image vector warehouse that is most similar to the target object image according to the cosine distance; determining a target object image according to the to-be-searched image and the segmentation mask corresponding to the search object image, comprising: selecting whether the search object image needs to consider surrounding object features; when the search object image needs to consider surrounding object features, performing blur processing on the to-be-searched image to determine a blurred image; determining an image in which an object part is clear and surrounding objects are blurred according to the blurred image, the to-be-searched image and the segmentation mask corresponding to the search object image; determining the target object image as the image in which the object part is clear and the surrounding objects are blurred; when the search object image does not need to consider surrounding object features, determining an image in which an object part is clear and surrounding objects are black according to the to-be-searched image and the segmentation mask corresponding to the search object image; finding out an image contour of the image in which the object part is clear and the surrounding objects are black, and then extracting the image in which the pixel is not 0 to obtain an image containing only an object; determining the target object image as the image containing only the object; when the search object image needs to consider surrounding objects, performing blur processing on the to-be-searched image to determine a blurred image, comprising: obtaining the horizontal and vertical coordinates of pixels of the to-be-searched image; setting the size of a Gaussian convolution kernel and obtaining the horizontal and vertical coordinates of the Gaussian convolution kernel; determining a blurred image according to the to-be-searched image, the horizontal and vertical coordinates of the pixels of the to-be-searched image, the size of the Gaussian convolution kernel and the horizontal and vertical coordinates of the Gaussian convolution kernel; determining a blurred image according to the to-be-searched image, the horizontal and vertical coordinates of the pixels of the to-be-searched image, the size of the Gaussian convolution kernel and the horizontal and vertical coordinates of the Gaussian convolution kernel, comprising: according to the formula Determine a blurred image Wherein x is the horizontal coordinate of the image pixel to be searched, y is the vertical coordinate of the image pixel to be searched, K is the Gaussian convolution kernel, w is the horizontal coordinate of the Gaussian convolution kernel, h is the vertical coordinate of the Gaussian convolution kernel, t is the size of the Gaussian convolution kernel, and k is the value of the parity of t. The blurred image is determined according to the segmentation mask corresponding to the blurred image, the image to be searched, and the search object image, and the image in which the object part is clear and the surrounding object is blurred is determined. according to the formula Image with clear object part and blurred surrounding objects wherein, is a segmentation mask corresponding to the search object image, is an image to be searched, is a blurred image, when the search object image does not need to consider the features of surrounding objects, determining an image with clear object part and blurred surrounding black according to the segmentation mask corresponding to the search object image and the image to be searched. according to the formula Image with clear object portion and dark background wherein, is a segmentation mask corresponding to the search object image, is the image to be searched.

2. The SAM based smart image search method according to claim 1, wherein, determining a cosine distance between each image in the image vector warehouse and the target object image according to the feature vector of the target object image and the feature vector of each image in the image vector warehouse, comprising: obtaining multi-dimensional feature values of the feature vector of the target object image; obtaining multi-dimensional feature values of the feature vector of each image in the image vector warehouse, wherein the dimension of the feature vector of each image in the image vector warehouse is consistent with the dimension of the feature vector of the target object image; determining the cosine distance between each image in the image vector warehouse and the target object image according to the multi-dimensional feature values of the feature vector of the target object image and the multi-dimensional feature values of the feature vector of each image in the image vector warehouse.

3. The SAM based smart image search method according to claim 2, wherein, determining the cosine distance between each image in the image vector warehouse and the target object image according to the multi-dimensional feature values of the feature vector of the target object image and the multi-dimensional feature values of the feature vector of each image in the image vector warehouse, comprising: according to the formula determining a cosine distance between the Ith image of the image vector repository and the target object image wherein v is a feature vector of the target object image, is a feature vector of the Ith image of the image vector repository, is an i-dimensional eigenvalue of the feature vector of the target object image, is an i-dimensional eigenvalue of the Ith image of the image vector repository, n is a dimension of the feature vector of the target object image and the feature vector of each image of the image vector repository, i≤n, and I, i and n are all positive integers.

4. The SAM based smart image search method as claimed in claim 1, wherein, determining the image most similar to the target object image in the image vector warehouse according to the cosine distance, comprising: sorting the cosine distance between each image in the image vector warehouse and the target object image in ascending order; the image ranked first is the image most similar to the target object image in the image vector warehouse.

Images