Image matching method, device, electronic device and storage medium

By combining local histogram dimensionality reduction and binary features on image blocks, the time-consuming problem of TreeCANN in high-resolution image processing is solved, and the efficiency and accuracy of image block matching are improved.

CN112581516BActive Publication Date: 2024-11-22YUANLI TUXIN (CHONGQING) TECHNOLOGY CO LTD
View PDF 10 Cites 0 Cited by

Patent Information

Application Number
CN202011384473.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2020-11-30
Publication Date
2024-11-22
Estimated Expiration
2040-11-30

AI Technical Summary

Technical Problem

In the existing image block matching technology, especially the TreeCANN method, in high-resolution image processing, the dimensionality reduction process is time-consuming and sensitive to high-dimensional data, resulting in a long preliminary construction process, which affects efficiency.

Method used

By characterizing the preset neighborhood of each cell in the image, a local histogram is obtained, and dimensionality reduction is performed to build a binary tree structure, and querying is combined with binary feature vectors to improve matching efficiency.

Benefits of technology

It reduces the sensitivity of KD-Tree to high-dimensional data, improves the efficiency and accuracy of image matching, and shortens the construction and query time.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN112581516B_ABST
    Figure CN112581516B_ABST
Patent Text Reader

Abstract

The present application provides an image matching method, device, electronic device and storage medium. The image matching method includes: performing a feature description on a preset neighborhood of each first pixel in a first image to obtain a first local histogram of the corresponding first block; performing dimensionality reduction on each first local histogram to obtain a first reduced-dimensionality histogram space; constructing a binary tree structure based on the first reduced-dimensionality histogram space; performing a feature description on a preset neighborhood of at least one second pixel in a second image to obtain a second local histogram of the corresponding second block; performing dimensionality reduction on each second local histogram to obtain a corresponding second reduced-dimensionality local histogram; querying the first block in the first image with the highest similarity to each of the second blocks based on each second reduced-dimensionality local histogram and the binary tree structure. The present application can improve the efficiency of image matching.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the technical field of computer image recognition, and in particular to an image matching method, device, electronic device and storage medium. Background Art

[0002] Image block matching technology is a hot topic in computer vision and other fields, and is widely used in image fusion, image super resolution, stereo vision and other fields. How to quickly and high-quality complete the most similar block matching between two images is the research focus of image block matching technology.

[0003] The mainstream technologies of existing image block matching mainly include PatchMatch, CSH, TreeCANN, etc. Among them, TreeCANN adopts the traditional idea of ​​dimensionality reduction and tree building, but the dimensionality reduction process is very time-consuming, and KDTree is very sensitive to building query binary trees for high-dimensional data, especially for high-resolution images, which is very time-consuming, resulting in a long tree building process in the early stage of this method.

[0004] There is currently no effective technical solution to the above problems. Summary of the invention

[0005] The purpose of the embodiments of the present application is to provide an image matching method, device, electronic device and storage medium, which can improve matching efficiency.

[0006] In a first aspect, an embodiment of the present application provides an image matching method, comprising:

[0007] Performing feature description on a preset neighborhood of each first pixel in the first image to obtain a first local histogram of a corresponding first image block, wherein each first image block is composed of a corresponding first pixel in the first image and a preset neighborhood of the first pixel;

[0008] Performing dimensionality reduction on each first local histogram to obtain a first dimensionality reduction histogram space;

[0009] Constructing a binary tree structure according to the first dimension reduction histogram space;

[0010] Performing feature description on a preset neighborhood of at least one second pixel in the second image to obtain a second local histogram corresponding to a second image block, wherein the second image block is composed of the corresponding second pixel in the second image and the preset neighborhood of the second pixel;

[0011] Performing dimensionality reduction on each second local histogram to obtain a corresponding second dimensionality-reduced local histogram;

[0012] A first image block in the first image having the highest similarity to each of the second image blocks is searched according to each of the second dimensionality reduction local histograms and the binary tree structure.

[0013] Optionally, in the image matching method described in the embodiment of the present application, the at least one second pixel includes every second pixel of the second image, or the at least one second pixel includes part of the second pixels of the second image.

[0014] Optionally, in the image matching method described in the embodiment of the present application, the feature description of the preset neighborhood of each first pixel in the first image to obtain the first local histogram of each corresponding first image block includes:

[0015] The formula hist(x)=F(x) is used to calculate the first local histogram of the first image block corresponding to each first pixel of the first image, where x represents a discrete grayscale value of [0, 255], and F(x) represents the frequency of occurrence of the discrete grayscale value x in the corresponding first image block.

[0016] Optionally, in the image matching method described in the embodiment of the present application, the step of reducing the dimension of each first local histogram to obtain a first reduced-dimensional histogram space includes:

[0017] The 256-dimensional description vector of each of the first local histograms is merged into an M-dimensional description vector to obtain a corresponding first reduced-dimensionality local histogram, wherein M is less than 256.

[0018] Optionally, in the image matching method described in the embodiment of the present application, the method further includes the following steps:

[0019] Performing a binary description on each of the first image blocks to obtain a corresponding first binary space, wherein the first binary space includes a binary feature vector of each of the first image blocks;

[0020] Performing a binary description on each of the second image blocks to obtain a corresponding second binary space, where the second binary space includes a binary feature vector of each of the second image blocks;

[0021] The querying, according to each of the second dimensionality reduction local histograms and the binary tree structure, for a first image block in the first image having the highest similarity to each of the second image blocks comprises:

[0022] A first image block having the greatest similarity to each of the second image blocks in the first image is searched according to the second reduced-dimensionality local histogram and the binary feature vector of each of the second image blocks, the binary tree structure, and the first binary space.

[0023] The embodiment of the present application further improves the accuracy of image matching by combining the reduced-dimensional histogram space with the binary feature vector of the binary space.

[0024] Optionally, in the image matching method described in the embodiment of the present application, querying the first image block having the greatest similarity to each of the second image blocks in the first image according to the second reduced-dimensionality local histogram and the binary feature vector of each second image block, the binary tree structure, and the first binary space includes:

[0025] Searching for a target node with the greatest similarity on the binary tree structure according to the second reduced-dimensionality local histogram of each second image block;

[0026] The binary feature vectors corresponding to a plurality of nodes within a preset range centered on the target node of each of the first image blocks are respectively compared with the binary feature vectors of the second image blocks to select the first image block with the highest similarity.

[0027] Optionally, in the image matching method described in the embodiment of the present application, comparing the binary feature vectors corresponding to multiple nodes within a preset range centered on the target node with the binary feature vectors of the second image block respectively to select the first image block with the highest similarity includes:

[0028] Compare the binary feature vectors corresponding to a plurality of nodes within a preset range centered on the target node of each of the first image blocks with the binary feature vectors of the second image blocks, and use the Hamming distance as a similarity function to screen out a plurality of candidate image blocks whose similarities are greater than a preset threshold;

[0029] The plurality of candidate image blocks are sorted by binary Hamming distance to select a candidate image block with the smallest binary Hamming distance as the first image block with the greatest similarity to the corresponding second image block.

[0030] Optionally, in the image matching method described in the embodiment of the present application, each of the first image blocks is described in binary to obtain a corresponding first binary space, and the first binary space includes a binary feature vector of each first image block, including:

[0031] The binary description of each of the first tiles is as follows:

[0032] When the pixel value of pixel i is less than the pixel value of pixel j, the pixel value of pixel i is binarized to 0;

[0033] When the pixel value of pixel i is greater than or equal to the pixel value of pixel j, the pixel value of pixel i is binarized to 1;

[0034] Among them, j∈Ni, i represents the pixel to be described in the first image, j represents the neighborhood pixel of pixel i, and Ni represents the neighborhood range of pixel i; when the local structure of each first block of the first image is described, the corresponding first binary space is obtained.

[0035] In a second aspect, an embodiment of the present application provides an image matching device, including:

[0036] A first acquisition module is configured to perform a feature description on a preset neighborhood of each first pixel in the first image to obtain a first local histogram of each corresponding first image block, wherein the first image block is composed of a corresponding first pixel in the first image and the preset neighborhood of the first pixel;

[0037] A first dimension reduction module, used for reducing the dimension of each first local histogram to obtain a first dimension reduction histogram space;

[0038] A construction module, used to construct a binary tree structure according to the first dimensionality reduction histogram space;

[0039] a second acquisition module, configured to perform a feature description on a preset neighborhood of at least one second pixel in the second image to acquire a second local histogram corresponding to a second image block, wherein the second image block is composed of the corresponding second pixel in the second image and the preset neighborhood of the second pixel;

[0040] A second dimension reduction module, used for reducing the dimension of each second local histogram to obtain a corresponding second dimension-reduced local histogram;

[0041] A query module is used to query the first image block in the first image for the highest similarity to each of the second image blocks according to each of the second dimensionality reduction local histograms and the binary tree structure.

[0042] In a third aspect, an embodiment of the present application provides an electronic device, comprising a processor and a memory, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the steps in the method provided in the first aspect are performed.

[0043] In a fourth aspect, an embodiment of the present application provides a storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the steps in the method provided in the first aspect are performed.

[0044] As can be seen from the above, the embodiment of the present application performs a feature description on the preset neighborhood of each first pixel in the first image to obtain a first local histogram of each corresponding first image block, wherein the first image block is composed of a corresponding first pixel in the first image and the preset neighborhood of the first pixel; performs dimensionality reduction on each first local histogram to obtain a first reduced-dimensionality histogram space; constructs a binary tree structure based on the first reduced-dimensionality histogram space; performs a feature description on the preset neighborhood of at least one second pixel in the second image to obtain a second local histogram corresponding to the second image block, wherein , the second image block is composed of the corresponding second pixel in the second image and the preset neighborhood of the second pixel; each second local histogram is reduced in dimension to obtain the corresponding second reduced-dimensional local histogram; according to each second reduced-dimensional local histogram and the binary tree structure, the first image block with the highest similarity to each second image block in the first image is queried; thereby realizing the search for the second image block with the highest similarity to the second image block. Since it adopts the method of reducing the dimension of the local histogram, the sensitivity of KD-Tree to the high-dimensional data structure is reduced, and the matching efficiency can be improved.

[0045] Other features and advantages of the present application will be described in the following description, and partly become apparent from the description, or be understood by practicing the embodiments of the present application. The purpose and other advantages of the present application can be realized and obtained by the structures specifically pointed out in the written description, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0047] Figure 1 A flowchart of an image matching method provided in an embodiment of the present application.

[0048] Figure 2 Another flowchart of the image matching method provided in an embodiment of the present application.

[0049] Figure 3 A first structural schematic diagram of the image matching device provided in an embodiment of the present application.

[0050] Figure 4 A second structural schematic diagram of the image matching device provided in an embodiment of the present application.

[0051] Figure 5A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0052] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0053] It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. At the same time, in the description of this application, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0054] Please refer to Figure 1 , Figure 1 is a flowchart of an image matching method in some embodiments of the present application, the method comprising the following steps:

[0055] S101. Performing feature description on a preset neighborhood of each first pixel in a first image to obtain a first local histogram of each corresponding first image block, wherein the first image block is composed of a corresponding first pixel in the first image and a preset neighborhood of the first pixel.

[0056] S102: Perform dimensionality reduction on each first local histogram to obtain a first dimensionality reduction histogram space.

[0057] S103: construct a binary tree structure according to the first dimensionality reduction histogram space.

[0058] S104, performing feature description on a preset neighborhood of at least one second pixel in the second image to obtain a second local histogram corresponding to a second image block, wherein the second image block is composed of a corresponding second pixel in the second image and a preset neighborhood of the second pixel.

[0059] S105 , performing dimensionality reduction on each second local histogram to obtain a corresponding second dimensionality-reduced local histogram.

[0060] S106: Query, according to each of the second dimensionality reduction local histograms and the binary tree structure, a first image block in the first image that has the highest similarity to each of the second image blocks.

[0061] In step S101, the first pixel is a pixel, which is the smallest unit constituting a digital image. The preset neighborhood may be an N*N neighborhood. The first local histogram is calculated based on the grayscale statistical information of each first image block.

[0062] Specifically, in some embodiments, step S101 specifically includes: using the formula hist(x)=F(x) to calculate the first local histogram of the first image block corresponding to each first pixel of the first image, wherein x represents a discrete grayscale value of [0, 255], and F(x) represents the frequency of occurrence of the discrete grayscale value x in the corresponding first image block.

[0063] In step S102, the 256-dimensional description vectors of each of the first local histograms are merged into an M-dimensional description vector to obtain a corresponding first reduced-dimensionality local histogram, wherein M is less than 256. Of course, it can be understood that M can be 64, and of course the 256-dimensional description vectors of the first local histograms can also be merged into description vectors of other dimensions, for example, M can also be equal to 128 or 32. The following formula can be used to achieve the merging of description vectors:

[0064] Where d represents the compression dimension 4. Through the histogram statistics and dimensionality reduction of the local image block, the first dimensionality reduction histogram space H of the first image is obtained. b The first dimension reduction histogram space H b A plurality of first dimension reduction histograms are included.

[0065] In step S103, the nearest element can be quickly found by constructing a binary tree structure. The binary tree structure KD-Tree is a high-dimensional space data structure. The first dimension-reduced histogram space H b To construct a binary tree structure KD-Tree.

[0066] Specifically, in some embodiments, step S103 includes the following sub-steps:

[0067] S1031, calculate the difference between the maximum vector value and the minimum vector value in each dimension of N (for example, 64) dimensions in the first dimensionality reduction histogram space, and filter out the dimension dim_max with the largest difference from the N dimensions; S1032, use the dimension where dim_max is located as the segmentation dimension of the local histogram vector space to divide the first dimensionality reduction histogram space H b The space H is divided into two subspacesb1 and H b2 Repeat step S1031 and step S1032 until the binary tree structure KD-Tree is created.

[0068] Wherein, in step S104, the second pixel is also a pixel, which is the smallest unit constituting a digital image. The preset neighborhood can be an N*N neighborhood. The second local histogram is calculated based on the grayscale statistical information of each second image. The second local histogram of the second image can be obtained in the same manner as step S101, so it is not repeated. Wherein, at least one second pixel includes each second pixel of the second image, or the at least one second pixel includes part of the second pixels of the second image.

[0069] In this step S105, each second local histogram is reduced in dimension in the same manner as step S102, and the ratio of dimensionality reduction is also the same, to obtain a second reduced dimensionality histogram space, and the second reduced dimensionality histogram space includes the second reduced dimensionality local histogram of each second block of the second image.

[0070] In step S106, the second local dimension reduction histogram of each second image block can be used to find the most similar node on the KD-Tree structure. The most similar node refers to the node with the smallest histogram Euclidean distance calculation formula, where each node corresponds to a second pixel. Here, the histogram Euclidean distance calculation formula is as follows:

[0071]

[0072] As can be seen from the above, the embodiment of the present application performs a feature description on the preset neighborhood of each first pixel in the first image to obtain a first local histogram of each corresponding first image block, wherein the first image block is composed of a corresponding first pixel in the first image and the preset neighborhood of the first pixel; performs dimensionality reduction on each first local histogram to obtain a first reduced-dimensionality histogram space; constructs a binary tree structure based on the first reduced-dimensionality histogram space; performs a feature description on the preset neighborhood of at least one second pixel in the second image to obtain a second local histogram corresponding to the second image block, wherein , the second image block is composed of the corresponding second pixel in the second image and the preset neighborhood of the second pixel; each second local histogram is reduced in dimension to obtain the corresponding second reduced-dimensional local histogram; according to each second reduced-dimensional local histogram and the binary tree structure, the first image block with the highest similarity to each second image block in the first image is queried; thereby realizing the search for the second image block with the highest similarity to the second image block. Since it adopts the method of reducing the dimension of the local histogram, the sensitivity of KD-Tree to the high-dimensional data structure is reduced, and the matching efficiency is improved.

[0073] Please refer to Figure 2 , Figure 2 1 is a flowchart of an image matching method in some other embodiments of the present application. In this embodiment, in order to improve the query accuracy of the most similar block, instead of using only the local histogram vector, the elements in the binary space of each first image and the second image are also used for comparison to improve the accuracy. The flowchart of the image matching method includes the following steps:

[0074] S201. Perform a feature description on a preset neighborhood of each first pixel in a first image to obtain a first local histogram of a corresponding first image block, wherein each first image block is composed of a corresponding first pixel in the first image and a preset neighborhood of the first pixel.

[0075] S202: Perform dimensionality reduction on each first local histogram to obtain a first dimensionality reduction histogram space.

[0076] S203: construct a binary tree structure according to the first dimensionality reduction histogram space.

[0077] S204 , performing binary description on each of the first image blocks to obtain a corresponding first binary space, where the first binary space includes a binary feature vector of each of the first image blocks.

[0078] S205. Perform a feature description on a preset neighborhood of at least one second pixel in the second image to obtain a second local histogram corresponding to a second image block, wherein the second image block is composed of a corresponding second pixel in the second image and a preset neighborhood of the second pixel.

[0079] S206 : Perform dimensionality reduction on each second local histogram to obtain a corresponding second dimensionality-reduced local histogram.

[0080] S207, performing binary description on each of the second image blocks to obtain a corresponding second binary space, where the second binary space includes a binary feature vector of each of the second image blocks;

[0081] S208 , searching the first image block in the first image for a first image block having the greatest similarity to each of the second image blocks according to the second reduced-dimensionality local histogram and the binary feature vector of each of the second image blocks, the binary tree structure, and the first binary space.

[0082] Among them, the steps S201 to S203 correspond to the steps S101 to S103, and the steps S205 to S206 correspond to the steps S104 to S105, so no detailed description is required.

[0083] In step S204, the structural features of the image block may be expressed by a binary descriptor. The binary description of each of the first image blocks is as follows:

[0084] When the pixel value of pixel i is less than the pixel value of pixel j, the pixel value of pixel i is binarized to 0;

[0085] When the pixel value of pixel i is greater than or equal to the pixel value of pixel j, the pixel value of pixel i is binarized to 1;

[0086] Wherein, j∈Ni, i represents the pixel to be described in the first image, j represents the neighboring pixel of pixel i, and Ni represents the neighborhood range of pixel i; when the local structure of each first block of the first image is described, the corresponding first binary space B is obtained. b .

[0087] In step S207, the same method as step S204 may be used to perform a binary description of each second block of the second image, so as to obtain a binary feature vector of each second block in the corresponding second binary space.

[0088] In step S208, the second dimensionality reduction local histogram of the second image block may be used for preliminary search, and then the binary feature vector of the image block may be used for accurate positioning, thereby obtaining the first image block with the greatest similarity to each second image block.

[0089] Specifically, step S208 includes: S2081, searching for the target node with the greatest similarity on the binary tree structure according to the second dimensionality reduction local histogram of each second image block; S2082, comparing the binary feature vectors corresponding to multiple nodes within a preset range centered on the target node with the binary feature vectors of the second image block, so as to select the first image block with the highest similarity.

[0090] Among them, in step S2081, the target node with the greatest similarity can be found by using a similarity comparison function, and specifically, the similarity comparison function uses the histogram Euclidean distance. The target node is the node with the smallest histogram Euclidean distance with the second image block to be matched. Here, the histogram Euclidean distance calculation formula is as follows:

[0091]

[0092] Among them, in this step S2082, the binary feature vector of the second image block to be matched can be used to calculate the Hamming distance with the binary feature vectors corresponding to the multiple nodes within the preset range, and the node with the smallest Hamming distance can be selected, so as to find the first image block that is most similar to the second image block to be matched. Specifically, this step S2082 includes: comparing the binary feature vectors corresponding to the multiple nodes within the preset range centered on the target node with the binary feature vector of the second image block, and using the Hamming distance as the similarity function to screen out multiple candidate image blocks with similarities greater than a preset threshold; sorting the multiple candidate image blocks by binary Hamming distance to select the candidate image block with the smallest binary Hamming distance as the first image block with the greatest similarity to the corresponding second image block.

[0093] If the at least one second pixel includes every second pixel of the second image, or the at least one second pixel includes multiple second pixels of the second image, then step S208 is repeated continuously so that the first image block with the greatest similarity to the second image block corresponding to each second pixel is found. Of course, if the at least one second pixel includes only one second pixel, then it only needs to be performed once.

[0094] It can be seen from the above that, based on the above embodiments, the embodiments of the present application further improve the accuracy of image matching by combining the reduced-dimensional histogram space with the binary space.

[0095] Please refer to Figure 3 , Figure 3 It is a structural diagram of an image matching device in some embodiments of the present application, wherein the image matching device comprises: a first acquisition module 301, a first dimensionality reduction module 302, a construction module 303, a second acquisition module 304, a second dimensionality reduction module 305 and a query module 306.

[0096] The first acquisition module 301 describes the characteristics of the preset neighborhood of each first pixel in the first image to obtain the first local histogram corresponding to the first image block, wherein the first image block is composed of a corresponding first pixel in the first image and the preset neighborhood of the first pixel. The first pixel is also a pixel point, which is the smallest unit that constitutes a digital image. The preset neighborhood can be an N*N neighborhood. The first local histogram is calculated based on the grayscale statistical information of each first image block.

[0097] Specifically, in some embodiments, the first acquisition module 301 is used to calculate the first local histogram of the first image block corresponding to each first pixel of the first image using the formula hist(x)=F(x), wherein x represents a discrete grayscale value of [0, 255], and F(x) represents the frequency of occurrence of the discrete grayscale value x in the corresponding first image block.

[0098] Wherein, the first dimensionality reduction module 302 is used to reduce the dimension of each first local histogram to obtain a first dimensionality reduction histogram space. The 256-dimensional description vector of each of the first local histograms is merged into an M-dimensional description vector to obtain the corresponding first dimensionality reduction local histogram. Of course, it can be understood that, for example, M can be equal to 64, 128 or 32. Wherein, the following formula can be used to realize the merging of description vectors:

[0099] Where d represents the compression dimension 4. Through the histogram statistics and dimensionality reduction of the local image block, the first dimensionality reduction histogram space H of the first image is obtained. b The first dimension reduction histogram space H b A plurality of first dimension reduction histograms are included.

[0100] The construction module 303 is used to construct a binary tree structure according to the first dimension reduction histogram space. The binary tree structure KD-Tree is a high-dimensional space data structure. The binary tree structure can be used to quickly find the nearest element. b Construct a binary tree structure KD-Tree.

[0101] The second acquisition module 304 is used to perform the feature description on a preset neighborhood of at least one second pixel in the second image to obtain a second local histogram corresponding to the second image block, wherein the second image block is composed of the corresponding second pixel in the second image and the preset neighborhood of the second pixel.

[0102] The second dimension reduction module 305 is used to reduce the dimension of each second local histogram to obtain a corresponding second dimension-reduced local histogram.

[0103] The query module 306 is used to query the first image block with the highest similarity to each of the second image blocks according to each of the second dimensionality reduction local histograms and the binary tree structure. The second local dimensionality reduction histogram of each second image block can be used to find the most similar node on the KD-Tree structure. The most similar node refers to the node with the smallest histogram Euclidean distance calculation formula, where each node corresponds to a second pixel. Here, the histogram Euclidean distance calculation formula is as follows:

[0104]

[0105] As can be seen from the above, the device provided by the present application performs a feature description on the preset neighborhood of each first pixel in the first image to obtain a first local histogram of each corresponding first image block, wherein the first image block is composed of a corresponding first pixel in the first image and the preset neighborhood of the first pixel; performs dimensionality reduction on each first local histogram to obtain a first reduced-dimensionality histogram space; constructs a binary tree structure based on the first reduced-dimensionality histogram space; performs a feature description on the preset neighborhood of at least one second pixel in the second image to obtain a second local histogram corresponding to the second image block, wherein the first image block is composed of a corresponding first pixel in the first image and the preset neighborhood of the first pixel; performs a dimensionality reduction on each first local histogram to obtain a first reduced-dimensionality histogram space; constructs a binary tree structure based on the first reduced-dimensionality histogram space; performs a feature description on the preset neighborhood of at least one second pixel in the second image to obtain a second local histogram corresponding to the second image block, The two blocks are composed of the corresponding second pixels in the second image and the preset neighborhood of the second pixels; each second local histogram is reduced in dimension to obtain the corresponding second reduced-dimensional local histogram; the first block in the first image with the highest similarity to each second block is queried according to each second reduced-dimensional local histogram and the binary tree structure; thereby realizing the search for the second block with the greatest similarity to the second block. Since it adopts the method of reducing the dimension of the local histogram, the sensitivity of KD-Tree to the high-dimensional data structure is reduced, and the tree can be quickly built and queried, thereby improving the matching efficiency.

[0106] Please refer to Figure 4 , Figure 4 is a schematic diagram of the structure of an image matching device in some embodiments of the present application. Figure 3 On the basis of the illustrated embodiment, a first description module 307 and a second description module 308 are added.

[0107] The first description module 307 performs binary description on each of the first image blocks to obtain a corresponding first binary space, and the first binary space includes a binary feature vector of each of the first image blocks.

[0108] The second description module 308 performs binary description on each of the second image blocks to obtain a corresponding second binary space, and the second binary space includes a binary feature vector of each of the second image blocks.

[0109] Correspondingly, the query module 306 is used to query the first image block with the greatest similarity to each of the second image blocks according to the second reduced-dimensionality local histogram and binary feature vector of each of the second image blocks, the binary tree structure, and the first binary space.

[0110] Please refer to Figure 5 , Figure 5This is a structural diagram of an electronic device provided in an embodiment of the present application. The present application provides an electronic device 4, including: a processor 401 and a memory 402. The processor 401 and the memory 402 are interconnected and communicate with each other through a communication bus 403 and / or other forms of connection mechanisms (not marked). The memory 402 stores a computer program executable by the processor 401. When the computing device is running, the processor 401 executes the computer program to execute the method in any optional implementation method of the above-mentioned embodiment.

[0111] The embodiment of the present application provides a storage medium, and when the computer program is executed by the processor, the method in any optional implementation of the above embodiment is executed. Wherein, the storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (Static Random Access Memory, referred to as SRAM), electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, referred to as EEPROM), erasable programmable read-only memory (Erasable Programmable Read Only Memory, referred to as EPROM), programmable read-only memory (Programmable Red-Only Memory, referred to as PROM), read-only memory (Read-Only Memory, referred to as ROM), magnetic memory, flash memory, disk or optical disk.

[0112] In the embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0113] In addition, the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0114] Furthermore, the functional modules in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

[0115] In this document, relational terms such as first and second, etc. are used merely to distinguish one entity or operation from another entity or operation, but do not necessarily require or imply any such actual relationship or order between these entities or operations.

[0116] The above description is only an embodiment of the present application and is not intended to limit the protection scope of the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. An image matching method, characterized in that: include: Performing feature description on a preset neighborhood of each first pixel in the first image to obtain a first local histogram corresponding to a first image block, wherein each first image block is composed of a corresponding first pixel in the first image and a preset neighborhood of the first pixel; Performing dimensionality reduction on each first local histogram to obtain a first dimensionality reduction histogram space; Constructing a binary tree structure according to the first dimension reduction histogram space; Performing feature description on a preset neighborhood of at least one second pixel in the second image to obtain a second local histogram corresponding to a second image block, wherein the second image block is composed of the corresponding second pixel in the second image and the preset neighborhood of the second pixel; Performing dimensionality reduction on each second local histogram to obtain a corresponding second dimensionality-reduced local histogram; Performing a binary description on each of the first image blocks to obtain a corresponding first binary space, wherein the first binary space includes a binary feature vector of each of the first image blocks; Performing a binary description on each of the second image blocks to obtain a corresponding second binary space, where the second binary space includes a binary feature vector of each of the second image blocks; searching for a target node with the greatest similarity on the binary tree structure according to the second dimensionality reduction local histogram of each second image block; The binary feature vectors corresponding to a plurality of nodes within a preset range centered on the target node of each of the first image blocks are respectively compared with the binary feature vectors of the second image blocks to select the first image block having the highest similarity to the second image block.

2. The image matching method according to claim 1, characterized in that: The at least one second pixel includes every second pixel of the second image, or the at least one second pixel includes some second pixels of the second image.

3. The image matching method according to claim 1, characterized in that: The step of performing feature description on a preset neighborhood of each first pixel in the first image to obtain a first local histogram of each corresponding first image block includes: The formula hist(x)=F(x) is used to calculate the first local histogram of the first image block corresponding to each first pixel of the first image, where x represents a discrete grayscale value of [0, 255], and F(x) represents the frequency of occurrence of the discrete grayscale value x in the corresponding first image block.

4. The image matching method according to claim 3, characterized in that: The step of reducing the dimension of each first local histogram to obtain a first reduced-dimensionality histogram space includes: The 256-dimensional description vector of each of the first local histograms is merged into an M-dimensional description vector to obtain a corresponding first reduced-dimensionality local histogram, wherein M is less than 256.

5. The image matching method according to claim 1, characterized in that: Comparing the binary feature vectors corresponding to a plurality of nodes within a preset range centered on the target node with the binary feature vectors of the second image block respectively to select the first image block with the highest similarity includes: Compare the binary feature vectors corresponding to a plurality of nodes within a preset range centered on the target node of each of the first image blocks with the binary feature vectors of the second image blocks, and use the Hamming distance as a similarity function to screen out a plurality of candidate image blocks whose similarities are greater than a preset threshold; The plurality of candidate image blocks are sorted by binary Hamming distance to select a candidate image block with the smallest binary Hamming distance as the first image block with the greatest similarity to the corresponding second image block.

6. The image matching method according to claim 1, characterized in that: The binary description is performed on each of the first image blocks to obtain a corresponding first binary space, wherein the first binary space includes a binary feature vector of each of the first image blocks, including: The binary description of each of the first tiles is as follows: When the pixel value of pixel i is less than the pixel value of pixel j, the pixel value of pixel i is binarized to 0; When the pixel value of pixel i is greater than or equal to the pixel value of pixel j, the pixel value of pixel i is binarized to 1; Among them, j∈Ni, i represents the pixel to be described in the first image, j represents the neighborhood pixel of pixel i, and Ni represents the neighborhood range of pixel i; when the local structure of each first block of the first image is described, the corresponding first binary space is obtained.

7. An image matching device, characterized in that: include: A first acquisition module is used to perform feature description on a preset neighborhood of each first pixel in the first image to obtain a first local histogram corresponding to a first image block, wherein each first image block is composed of a corresponding first pixel in the first image and the preset neighborhood of the first pixel; A first dimension reduction module, used for reducing the dimension of each first local histogram to obtain a first dimension reduction histogram space; A construction module, used to construct a binary tree structure according to the first dimensionality reduction histogram space; a second acquisition module, configured to perform a feature description on a preset neighborhood of at least one second pixel in the second image to acquire a second local histogram corresponding to a second image block, wherein the second image block is composed of the corresponding second pixel in the second image and the preset neighborhood of the second pixel; A second dimension reduction module, used for reducing the dimension of each second local histogram to obtain a corresponding second dimension-reduced local histogram; A first description module, configured to perform a binary description on each of the first image blocks to obtain a corresponding first binary space, wherein the first binary space includes a binary feature vector of each of the first image blocks; A second description module, configured to perform a binary description on each of the second image blocks to obtain a corresponding second binary space, wherein the second binary space includes a binary feature vector of each of the second image blocks; Query module for: Searching for a target node with the greatest similarity on the binary tree structure according to the second reduced-dimensionality local histogram of each second image block; The binary feature vectors corresponding to a plurality of nodes within a preset range centered on the target node of each of the first image blocks are respectively compared with the binary feature vectors of the second image blocks to select the first image block having the highest similarity to the second image block.

8. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the method according to any one of claims 1 to 6 is executed.

9. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is executed.

Citation Information

Patent Citations

  • Information processing apparatus, information processing method, and program

    CN102081497A

  • Similar picture identification method and similar picture identification device

    CN102622366A

  • Page matching method and device

    CN104111960A

  • Image ordering model based on convolutional neural network and spatial pyramid matching

    CN105718960A

  • Picture detection method and device

    CN106294577A