Intelligent similar image recognition and retrieval method and system

The Super Point algorithm extracts feature points and vector databases combined with HNSW and LightGlue algorithms, and solves the accuracy problem of image similarity search under light and angle changes, achieving efficient and accurate similar image retrieval.

CN119068267AInactive Publication Date: 2024-12-03SHENZHEN AIMALL TECHNOLOGY CO LTD

Patent Information

Application Number
CN202411556209.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-11-04
Publication Date
2024-12-03
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

When existing image similarity search technology deals with light changes and angle changes, feature extraction is difficult to adapt, resulting in low similarity calculation accuracy and difficulty in maintaining high cosine similarity under large angle changes.

Method used

Super Point algorithm is used to extract 512 feature points from the image, each feature point generates a 256-dimensional description vector, builds a vector database, and pre-retrieval and LightGlue algorithm are used to match it to improve retrieval efficiency and accuracy.

Benefits of technology

By extracting multiple feature points and vectorized features, combined with HNSW and LightGlue algorithms, the accuracy and efficiency of image retrieval is significantly improved, and similar images can be quickly found in large-scale image databases, reducing the number of images to be processed subsequently, and improving recall rate and matching accuracy.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119068267A_ABST
    Figure CN119068267A_ABST
Patent Text Reader

Abstract

The invention provides an intelligent similar image recognition and retrieval method and system. The method comprises the following steps: extracting key feature points from each image by using a Super Point algorithm; feature vectors are constructed, and a 256-dimensional description vector is generated for each feature point and used for representing local features of the feature points. According to the method, 512 feature points, 216-dimensional feature vectors corresponding to each feature point and vectorization features of [512 * 256] in total are extracted from a picture by using a Super Point algorithm, retrieval is carried out from the dimension of the points in a vector database, vote counting is carried out on the picture to which the point retrieved from each point belongs, the picture matched with the most points in the vector database is taken out, and the picture matched with the most points in the vector database is obtained. The method comprises the following steps of: screening candidate images to obtain topN images, matching the screened candidate images by using a LightGlue algorithm to obtain similar scene images, so that the pre-retrieved images are matched with higher precision, and the highest retrieval efficiency and precision are obtained under the condition of large-scale retrieval data volume.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to a retrieval method and system, in particular to an intelligent similar image recognition and retrieval method and system, belonging to the technical field of image similarity retrieval. Background Art

[0002] Image similarity retrieval refers to the process of analyzing and processing images through computer technology to retrieve images similar to the query image from a large-scale image database. This process mainly relies on the calculation of similarity between images, and the degree of similarity between them is determined by comparing the features of the images.

[0003] Image similarity retrieval is a retrieval technology based on image content. It can quickly and accurately search large-scale image databases by extracting image features and calculating similarity. With the continuous development of technologies such as deep learning, the accuracy and efficiency of image similarity retrieval will be further improved and play an important role in more fields. With the development of deep learning, automatic feature extraction methods based on deep learning have gradually become mainstream. For example, convolutional neural networks can learn more advanced and abstract feature representations from images.

[0004] The existing image scene similarity basically uses convolutional image algorithms such as CNN, efficient net or other algorithms to extract a 512-dimensional feature from the entire image and perform similarity measurement calculations with other images. However, this method extracts a feature from the entire image and is often difficult to adapt to various changes between images: such as light changes, angle changes, and so on. Under large angle changes, such features have only very low cosine similarity. Therefore, an intelligent similarity image recognition and retrieval method and system are proposed. Summary of the invention

[0005] In view of this, the present invention provides an intelligent similar image recognition and retrieval method and system to solve or alleviate the technical problems existing in the prior art and at least provide a beneficial option.

[0006] The technical solution of the embodiment of the present invention is implemented as follows: an intelligent similar image recognition and retrieval method comprises the following steps: Use Super Point algorithm to extract key feature points from each image; Construct feature vectors and generate 256-dimensional description vectors for each feature point to represent the local features of the feature point; Construct a vector database and store the feature points of the image and their description vectors in the vector database; Use the HNSW algorithm to pre-search in the vector database and find candidate images based on the description vectors of feature points; The LightGlue algorithm is used to match the screened candidate images to obtain similar scene images.

[0007] Further preferably, the Super Point algorithm extracts 512 feature points from the image, each feature point corresponds to a 256-dimensional feature vector, and the Super Point algorithm extracts [512*256] vectorized features from the image.

[0008] Further preferably, a retrieval and query mechanism is provided in the vector database, and the vector database supports the retrieval of large-scale image data.

[0009] Further preferably, when extracting key feature points, the Super Point algorithm includes the following steps: Input the image into the Super Point pre-trained network; The feature point detection and extraction module in the network identifies key locations in the image; For each detected feature point, a 256-dimensional description vector is generated to describe the local features around the feature point; Then the location information of the feature points and the corresponding description vector are output.

[0010] An intelligent similar image recognition and retrieval system, comprising a preprocessing module, a feature point detection and extraction module, a vector database, a pre-retrieval module, a screening module and a matching module; The signal end of the preprocessing module is connected to the signal end of the feature point detection and extraction module, the signal end of the vector database is connected to the signal end of the feature point detection module and the signal end of the pre-retrieval module respectively, and the signal end of the screening module is connected to the signal end of the matching module and the signal end of the pre-retrieval module respectively; The preprocessing module is used to preprocess the input image, including resizing and normalization.

[0011] Further preferably, the feature point detection and extraction module is used to extract key feature points from each image; The vector database is used to store feature points of an image and their description vectors.

[0012] Further preferably, the pre-retrieval module is used to pre-retrieve in the vector database using the HNSW algorithm to find the candidate image according to the description vector of the feature point; The screening module is used to perform preliminary screening on the candidate images obtained by pre-retrieval and remove irrelevant images.

[0013] Further preferably, the matching module is used to match the screened candidate images to obtain similar scene images.

[0014] Since the embodiment of the present invention adopts the above technical solution, it has the following advantages: the present invention uses the SuperPoint algorithm to extract 512 feature points from the picture, and the 216-dimensional feature vector corresponding to each feature point, with a total of [512 * 256] vectorized features, that is, a picture has 512 points, each point corresponds to a 256-dimensional vector, all stored image feature points and their description vectors are stored in a vector database, and then the HNSW algorithm is used to search the vector database from the point dimension, and the picture to which each retrieved point belongs is voted to obtain the topN pictures, and then the LightGlue algorithm is used to match the screened candidate images to obtain similar scene images, thereby achieving higher-precision matching of the pre-retrieved pictures.

[0015] The above summary is for illustrative purposes only and is not intended to be limiting in any way. In addition to the illustrative aspects, embodiments and features described above, further aspects, embodiments and features of the present invention will be readily apparent by reference to the accompanying drawings and the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0017] Figure 1 A flowchart of the steps of the intelligent similar image recognition and retrieval method of the present invention; Figure 2 It is a structural diagram of the intelligent similar image recognition and retrieval system of the present invention; Figure 3 The figure is a workflow diagram of the intelligent similar image recognition and retrieval system of the present invention. DETAILED DESCRIPTION

[0018] In the following, only some exemplary embodiments are briefly described. As those skilled in the art will appreciate, the described embodiments may be modified in various ways without departing from the spirit or scope of the present invention. Therefore, the drawings and descriptions are considered to be exemplary and non-restrictive in nature.

[0019] The embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0020] like Figure 1-Figure 3As shown, an embodiment of the present invention provides an intelligent similar image recognition and retrieval method, comprising the following steps: Use Super Point algorithm to extract key feature points from each image; Construct feature vectors and generate 256-dimensional description vectors for each feature point to represent the local features of the feature point; Construct a vector database and store the feature points of the image and their description vectors in the vector database; Using the HNSW algorithm to pre-search in the vector database and find candidate images based on the description vectors of feature points can significantly reduce the number of images to be processed later, improve retrieval efficiency, and maintain a high recall rate; The LightGlue algorithm is used to match the screened candidate images to obtain similar scene images.

[0021] In one embodiment, the Super Point algorithm extracts 512 feature points from the image, each feature point corresponds to a 256-dimensional feature vector, and the Super Point algorithm extracts [512*256] vectorized features from the image.

[0022] In one embodiment, a retrieval and query mechanism is provided in the vector database, and the vector database supports the retrieval of large-scale image data.

[0023] In one embodiment, the Super Point algorithm includes the following steps when extracting key feature points: Input the image into the Super Point pre-trained network; The feature point detection and extraction module in the network identifies key locations in the image; For each detected feature point, a 256-dimensional description vector is generated to describe the local features around the feature point, which is convenient for subsequent similarity calculation and retrieval; Then output the location information of the feature points and the corresponding description vector; Super Point is a feature point detection and descriptor extraction method based on deep learning. It can capture local significant features in images and generate a series of feature points with position information and description vectors. These points are sensitive to key information in the image and are robust to changes in lighting and viewing angle.

[0024] In one embodiment, the HNSW algorithm performs a pre-search step in a vector database; By using the HNSW algorithm, we can search the vector database from the point dimension and vote for the pictures to which each point belongs to, and get the top N pictures. This can improve the search efficiency. For example, we can use the HNSW algorithm to pre-search 100 similar pictures from 100,000 pictures. This step only takes 1 second, which is fast, but has low accuracy. The HNSW algorithm is mainly used for efficient pre-retrieval in vector databases, and quickly finds candidate images similar to the query image by building a hierarchical graph structure.

[0025] In one embodiment, the LightGlue algorithm matches the screened candidate images; LightGlue is an image matching algorithm based on deep learning. It can consider the global and local information of the image. By comprehensively considering the multi-faceted features of the image, it can improve the accuracy and robustness of the matching. Use the LightGlue algorithm to match the obtained pictures, and finally get truly similar scene pictures. The LightGlue algorithm has high accuracy in the matching process; LightGlue similarity calculation method, including: Use sigmoid function for normalization; The project code is: php Copy code function calculate_score(match_num, thresh=0.5, norm_pts_num=512,coeff=10): orig_score = ((match_num + 186) / norm_pts_num - 0.5) * coeff score = 1 / (1 + exp(-orig_score)) if score > thresh: isSame = True else: isSame = False return score, isSame in: match_num is the number of matching points, that is, the number of matching points between two sets of data (such as images or feature point sets).

[0026] Thresh is the score threshold for determining whether two objects are identical (the default value is 0.5). When the score is higher than this value, the two objects are considered to be identical.

[0027] norm_pts_num is the normalized number of feature points (the default value is 512), which is used to standardize the number of matching points; coeff is the coefficient for adjusting the score range, the default value is 10, which affects the range and sensitivity of the score; The function calculate_score calculates a matching score based on the input matching point number match_num, and based on the score, determines whether two objects (such as images or feature point sets) are the same (indicated by the Boolean value isSame).

[0028] The function process is: orig_score=((match_num+186) / norm_pts_num-0.5)*coeff This step is to normalize the number of matching points and add a bias of 186, then subtract 0.5, and finally multiply by coeff to expand the range of scores. The bias of 186 is to adjust the number of matching points to a distribution that is more conducive to classification; score=1 / (1+np.exp(-orig_score)) In this step, the sigmoid function is used to convert orig_score to the range of 0 to 1. The sigmoid function is often used to normalize values ​​to between [0, 1] and can handle nonlinear changes; According to the calculated score, it is compared with the threshold thresh. If score>thresh, isSame is True, indicating that the two objects are the same; otherwise isSame is False.

[0029] in, score is the matching score, which indicates the probability value of the similarity between two objects, ranging from 0 to 1; isSame is a Boolean value, indicating whether two objects are the same based on their scores.

[0030] The purpose of this function is to calculate the similarity score based on the number of matching points entered, and to determine whether the two sets of feature points belong to the same object based on the set threshold. By combining the HNSW algorithm and the LightGlue algorithm to retrieve similar images, it has a relatively high matching speed and efficiency, and also has excellent matching effects.

[0031] An intelligent similar image recognition and retrieval system, comprising a preprocessing module, a feature point detection and extraction module, a vector database, a pre-retrieval module, a screening module and a matching module; The signal end of the preprocessing module is connected to the signal end of the feature point detection and extraction module, the signal end of the vector database is connected to the signal end of the feature point detection module and the signal end of the pre-retrieval module respectively, and the signal end of the screening module is connected to the signal end of the matching module and the signal end of the pre-retrieval module respectively; The preprocessing module is used to preprocess the input image, including resizing and normalization, to ensure the consistency and efficiency of subsequent processing, improve the robustness and accuracy of the algorithm, and reduce errors caused by differences in image size or format.

[0032] In one embodiment, the feature point detection and extraction module is used to extract key feature points from each image; The vector database is used to store the feature points of the image and its description vector for fast retrieval. The vector database can implement efficient indexing and query mechanisms and support fast retrieval of large-scale image data.

[0033] In one embodiment, the pre-retrieval module is used to pre-retrieve in the vector database using the HNSW algorithm to find candidate images according to the description vectors of the feature points. The pre-retrieval module can significantly reduce the number of images to be processed subsequently, improve the retrieval efficiency, and maintain a high recall rate. The screening module is used to perform preliminary screening on the candidate images obtained by pre-retrieval and remove irrelevant images to further improve the accuracy of retrieval and reduce the amount of calculation for subsequent processing.

[0034] In one embodiment, the matching module is used to match the screened candidate images to obtain similar scene images. The matching module has a built-in LightGlue algorithm that can consider the global and local information of the image. By comprehensively considering various aspects of the image, the accuracy and robustness of the matching can be improved. The matching module also calculates a similarity score for each pair of matched images, based on the matching number and matching quality of feature points, which can quantify the similarity between images for subsequent sorting and selection.

[0035] When the present invention is working: the present invention first uses the Super Point algorithm to extract 512 feature points from the picture, and the 216-dimensional feature vector corresponding to each feature point, with a total of [512 * 256] vectorized features, that is, a picture has 512 points, each point corresponds to a 256-dimensional vector, all stored image feature points and their description vectors are stored in a vector database, and then the HNSW algorithm is used to search from the point dimension in the vector database, and the pictures to which each retrieved point belongs are voted to obtain the topN pictures, and then the LightGlue algorithm is used to match the screened candidate images, obtain similar scene images, and achieve higher-precision matching of the pre-retrieved pictures.

[0036] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of various changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. An intelligent similar image recognition and retrieval method, characterized in that: The following steps are involved: Use Super Point algorithm to extract key feature points from each image; Construct feature vectors and generate 256-dimensional description vectors for each feature point to represent the local features of the feature point; Construct a vector database and store the feature points of the image and their description vectors in the vector database; Use the HNSW algorithm to pre-search in the vector database and find candidate images based on the description vectors of feature points; The Light Glue algorithm is used to match the screened candidate images to obtain similar scene images.

2. The intelligent similar image recognition and retrieval method according to claim 1, characterized in that: The SuperPoint algorithm extracts 512 feature points from the image, each feature point corresponds to a 256-dimensional feature vector, and the Super Point algorithm extracts [512*256] vectorized features from the image.

3. The intelligent similar image recognition and retrieval method according to claim 2, characterized in that: The vector database is provided with a retrieval and query mechanism, and the vector database supports the retrieval of large-scale image data.

4. The intelligent similar image recognition and retrieval method according to claim 1, characterized in that: The SuperPoint algorithm includes the following steps when extracting key feature points: Input the image into the Super Point pre-trained network; The feature point detection and extraction module in the network identifies key locations in the image; For each detected feature point, a 256-dimensional description vector is generated to describe the local features around the feature point; Then the location information of the feature points and the corresponding description vector are output.

5. An intelligent similar image recognition and retrieval system, applied to the intelligent similar image recognition and retrieval method according to any one of claims 1 to 4, characterized in that: It includes a preprocessing module, a feature point detection and extraction module, a vector database, a pre-retrieval module, a screening module and a matching module; The signal end of the preprocessing module is connected to the signal end of the feature point detection and extraction module, the signal end of the vector database is connected to the signal end of the feature point detection module and the signal end of the pre-retrieval module respectively, and the signal end of the screening module is connected to the signal end of the matching module and the signal end of the pre-retrieval module respectively; The preprocessing module is used to preprocess the input image, including resizing and normalization.

6. The intelligent similar image recognition and retrieval system according to claim 5, characterized in that: The feature point detection and extraction module is used to extract key feature points from each image; The vector database is used to store feature points of an image and their description vectors.

7. The intelligent similar image recognition and retrieval system according to claim 6, characterized in that: The pre-retrieval module is used to pre-retrieve in the vector database using the HNSW algorithm and find the candidate image according to the description vector of the feature point; The screening module is used to perform preliminary screening on the candidate images obtained by pre-retrieval and remove irrelevant images.

8. The intelligent similar image recognition and retrieval system according to claim 7, characterized in that: The matching module is used to match the screened candidate images to obtain similar scene images.

Citation Information

Patent Citations

  • Cloud image recognition method based on vocabulary tree retrieval and similarity verification

    CN106874445A

  • Spatial failure satellite loopback detection method based on improved RANSAC verification

    CN114882254A

  • Feature matching method based on iterative matching thought

    CN117036744A

  • Indoor visual loopback detection method based on point-line feature global descriptor

    CN118038239A

  • Visual positioning method and electronic equipment

    CN118351181A

Cited By

  • Image processing method and device, equipment, medium and product

    CN120873218A

  • Intelligent registration method and system for batch warehouse-in and warehouse-out information of medical consumables

    CN122177392A