Image matching method and apparatus

Abstract

The embodiment of the present application provides a kind of image matching method and device, it is related to computer vision field, method includes: the image feature point of two images to be matched is determined, according to the image feature point, coordinate clustering is carried out, and according to coordinate clustering result, the image to be matched is divided into multiple image regions;The importance of each image region of the image to be matched is sorted, and the similarity of two image regions of the two images to be matched is determined according to the feature comparison of two image regions in order according to importance order;According to the similarity of the two image regions, whether the two images to be matched is determined;The present application can effectively improve image retrieval precision and matching efficiency.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of computer vision, and can also be used in the field of finance, and particularly relates to an image matching method and device. BACKGROUND

[0002] With the popularity of smart phones and digital products, users start to use digital products to take pictures and share them on some Internet social websites with some geographical tags, which makes the Internet image data information grow explosively. At the same time, the demand for image retrieval is also growing.

[0003] Image retrieval refers to inputting key information of an image to retrieve relevant images on the Internet. The key information includes tags of the image or the image itself, and the most convenient way for users is to upload the image directly to the Internet to query relevant images of the local image, such as a user wants to know about a building, can directly take a picture of the building, and then compare the image features with a large database on the Internet to finally obtain the information the user wants. However, with the increasing demand of users for image retrieval accuracy and speed, some traditional image retrieval methods cannot meet the needs of users. SUMMARY

[0004] In view of the problems in the prior art, the present application provides an image matching method and device, which can effectively improve the image retrieval accuracy and matching efficiency.

[0005] In order to solve at least one of the above problems, the present application provides the following technical solutions:

[0006] In a first aspect, the present application provides an image matching method, comprising:

[0007] determining image feature points of two images to be matched, performing coordinate clustering according to the image feature points, and dividing the images to be matched into a plurality of image regions according to the coordinate clustering results;

[0008] sequentially performing feature comparison on two image regions of the two images to be matched according to the importance order to determine the similarity of the two image regions;

[0009] determining whether the two images to be matched are matched according to the similarity of the two image regions.

[0010] Further, the determination of the image feature points of the two images to be matched comprises:

[0011] finding extreme points of the images to be matched in different scale spaces according to a preset scale invariant feature transformation algorithm to obtain the image feature points of the images to be matched.

[0012] Further, the coordinate clustering according to the image feature points, and dividing the to-be-matched images into multiple image regions according to the coordinate clustering result, comprises:

[0013] performing coordinate clustering on the coordinates of the image feature points according to a preset mean shift clustering algorithm;

[0014] determining multiple image regions according to the result of the coordinate clustering.

[0015] Further, the importance sorting of the multiple image regions of each to-be-matched image comprises:

[0016] performing importance sorting on the image regions according to the number of image feature points contained in each image region after the division.

[0017] Further, the importance sorting of the multiple image regions of each to-be-matched image further comprises:

[0018] performing importance sorting on the image regions according to the position distance between each image region after the division and the image center of the to-be-matched image.

[0019] Further, the feature comparison of the two image regions of the two to-be-matched images according to the importance order to determine the similarity of the two image regions comprises:

[0020] determining whether the two image regions of the two to-be-matched images have the same visual attribute according to the importance order in sequence;

[0021] determining the similarity of the two image regions according to the number of the same visual attributes possessed by the two image regions and the spatial descriptor of the two image regions.

[0022] In a second aspect, the present application provides an image matching device, comprising:

[0023] an image region division module, configured to determine image feature points of two to-be-matched images, perform coordinate clustering according to the image feature points, and divide the to-be-matched images into multiple image regions according to the coordinate clustering result;

[0024] a significant region determination module, configured to perform importance sorting on the multiple image regions of each to-be-matched image, perform feature comparison of two image regions of the two to-be-matched images according to the importance order in sequence, and determine the similarity of the two image regions;

[0025] an image matching module, configured to determine whether the two to-be-matched images match according to the similarity of the two image regions.

[0026] Further, the image region division module comprises:

[0027] The feature point determination unit is configured to find extreme points of the to-be-matched image in different scale spaces according to a preset scale-invariant feature transformation algorithm, and obtain image feature points of the to-be-matched image.

[0028] Further, the image region division module further comprises:

[0029] The coordinate clustering unit is configured to perform coordinate clustering on the coordinates of the image feature points according to a preset mean shift clustering algorithm.

[0030] The image division unit is configured to determine a plurality of image regions according to a result of the coordinate clustering.

[0031] Further, the salient region determination module comprises:

[0032] The number ordering unit is configured to perform importance ordering on the image regions according to the number of image feature points contained in each of the image regions.

[0033] Further, the salient region determination module further comprises:

[0034] The distance ordering unit is configured to perform importance ordering on the image regions according to a position distance between each of the image regions and a center of the to-be-matched image.

[0035] Further, the salient region determination module further comprises:

[0036] The same attribute judging unit is configured to determine whether two image regions of the two to-be-matched images have the same visual attribute in the order of importance.

[0037] The similarity calculating unit is configured to determine a similarity of the two image regions according to the number of the same visual attributes possessed by the two image regions and spatial descriptors of the two image regions.

[0038] In a third aspect, the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the image matching method.

[0039] In a fourth aspect, the present application provides a computer readable storage medium, which stores a computer program executable by a processor to implement the steps of the image matching method.

[0040] In a fifth aspect, the present application provides a computer program product, comprising a computer program / instruction executable by a processor to implement the steps of the image matching method.

[0041] It can be seen from the above technical solution that the present application provides an image matching method and device, which divides the image area into two parts, "salient" and "non-salient", and compares the image features of the significant area, thereby greatly reducing the number of image features, which is conducive to improving image retrieval accuracy and image retrieval matching efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0043] Figure 1 This is one of the flowcharts of the image matching method in the embodiment of the present application;

[0044] Figure 2 This is a second flow chart of the image matching method in an embodiment of the present application;

[0045] Figure 3 This is the third flow chart of the image matching method in the embodiment of the present application;

[0046] Figure 4 This is one of the structural diagrams of the image matching device in the embodiment of the present application;

[0047] Figure 5 This is the second structural diagram of the image matching device in the embodiment of the present application;

[0048] Figure 6 This is the third structural diagram of the image matching device in the embodiment of the present application;

[0049] Figure 7 This is the fourth structural diagram of the image matching device in the embodiment of the present application;

[0050] Figure 8 This is the fifth structural diagram of the image matching device in the embodiment of the present application;

[0051] Figure 9 This is the sixth structural diagram of the image matching device in the embodiment of the present application;

[0052] Figure 10 Schematic diagram of the structure of the electronic device in the embodiment of the present application. DETAILED DESCRIPTION

[0053] To make the purposes, technical solutions, and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some but not all of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative effort should fall within the protection scope of the present application.

[0054] The acquisition, storage, use, processing, and the like of data in the technical solutions of the present application comply with relevant provisions of national laws and regulations.

[0055] In view of the problems of low precision and speed of image retrieval in the prior art, the present application provides an image matching method and device, which divides an image into two parts of "salient" and "non-salient" through division of an image region, compares image features of a salient region, and thus greatly reduces the number of image features, which is conducive to improving image retrieval precision and image retrieval matching efficiency.

[0056] In order to effectively improve image retrieval precision and matching efficiency, an embodiment of an image matching method is provided in the present application, as shown in Figure 1 , and the image matching method specifically includes the following contents.

[0057] Step S101: determining image feature points of two images to be matched, performing coordinate clustering according to the image feature points, and dividing the images to be matched into multiple image regions according to the coordinate clustering result.

[0058] It can be understood that, since image retrieval / image matching is a similarity discrimination problem rather than an identification problem, the image does not need to be accurately segmented in this particular application field compared with image segmentation performed in object identification.

[0059] Optionally, in order to determine representative image feature points from the images to be matched, the present application can use a scale-invariant feature transform (SIFT) algorithm to determine extreme points of the two images to be matched in different scale spaces and take them as local image features, i.e., image feature points.

[0060] Optionally, the feature of region division is that the obtained region does not need to accurately match the contour boundary of a semantic entity in the original image, and thus an image content representation method based on a salient region can be used to avoid the difficulty caused by accurate image segmentation, and the distribution of image feature points itself can be used to mine image regions. Therefore, the present application can use a mean shift clustering method to perform coordinate clustering on image feature point coordinates to obtain multiple image regions.

[0061] Step S102: sorting the image regions of each image to be matched according to importance, and comparing the features of the image regions of the two images to be matched in order according to the importance to determine the similarity of the image regions.

[0062] Optionally, some regions are important regions expressing the theme of the image, and some regions are secondary regions supplementing and enriching the content of the image, which have different importance. Therefore, the application can distinguish the difference in importance when describing the image content, that is, sorting the image regions of each image to be matched according to importance.

[0063] Optionally, after sorting the image regions of the two images to be matched according to importance, the application can compare the features of the image regions in order according to the importance, for example, comparing the features of the image region with the highest importance of one image to be matched with the image region with the highest importance of the other image to be matched, and determining the similarity of the two image regions according to the comparison result.

[0064] Optionally, the similarity of the two image regions can be calculated based on the same visual attributes and the spatial descriptors of the two image regions.

[0065] Thus, the visual attributes of each image feature point are summarized into an index table. If two image regions (for example, a region of a query image and a region of an image set) of two images to be matched contain the same visual attributes, the two image regions are a pair of matched regions (RP). The similarity of the two image regions can be calculated by the following formula:

[0066]

[0067] In the formula, m represents the number of matched sub-region pairs in the two image regions, SD and SR represent the spatial descriptors of the four visual attributes in a pair of sub-region pairs, and the higher the score of MS, the more similar the two image regions.

[0068] Step S103: determining whether the two images to be matched are matched according to the similarity of the two image regions.

[0069] Optionally, the similarity of the two images to be matched can be determined according to the similarity of the two image regions calculated above, for example, the similarity expression is as follows:

[0070]

[0071] In the formula, S is the similarity of the two images to be matched, MS is the similarity of the two image regions, weight is the weight, and F is the number of similarities.

[0072] Therefore, two images to be matched whose similarity is higher than the threshold can be considered to be matched successfully.

[0073] From the above description, it can be seen that the image matching method provided in the embodiment of the present application can divide the image area into two parts, "salient" and "non-salient", and compare the image features of the significant area, thereby greatly reducing the number of image features, which is conducive to improving image retrieval accuracy and image retrieval matching efficiency.

[0074] In order to accurately extract the image feature points of the image to be matched, in one embodiment of the image matching method of the present application, the above step S101 may further specifically include the following content:

[0075] The extreme points of the image to be matched in different scale spaces are found according to a preset scale-invariant feature conversion algorithm to obtain image feature points of the image to be matched.

[0076] Optionally, in order to determine representative image feature points from the images to be matched, the present application can use the scale-invariant feature transform (SIFT) algorithm to determine the extreme points of the two images to be matched in different scale spaces, and use them as local image features, i.e., image feature points.

[0077] In order to accurately divide the image to be matched into multiple image regions, in one embodiment of the image matching method of the present application, see Figure 2 , the above step S101 may further specifically include the following contents:

[0078] Step S201: performing coordinate clustering on the coordinates of the image feature points according to a preset mean shift clustering algorithm.

[0079] Step S202: determining a plurality of image regions according to the coordinate clustering result.

[0080] Optionally, the characteristic of region division is that the obtained region is not required to accurately match the contour boundary of the semantic entity in the original image. Therefore, the image content representation method based on salient regions can be used to avoid the difficulties brought by precise image segmentation, and the image region can be mined with the help of the distribution of image feature points themselves. Therefore, the present application can adopt the mean shift clustering method to perform coordinate clustering on the image feature point coordinates to obtain multiple image regions.

[0081] In order to accurately determine the representative region among the multiple image regions, in one embodiment of the image matching method of the present application, the above step S102 may further specifically include the following contents:

[0082] The importance of each image region is sorted according to the number of image feature points contained in each divided image region.

[0083] Optionally, the number of image feature points contained in each image region is an important factor affecting the region weight, and therefore the application can sort the image regions according to the number of image feature points contained in each image region.

[0084] To accurately determine the representative region in the multiple image regions, in an embodiment of the image matching method of the application, the step S102 can further include the following content:

[0085] The image regions are sorted according to the position distance between each image region and the image center of the image to be matched.

[0086] Optionally, the closer the distance between the position of the image region and the position of the image center, the greater the importance factor, because for most images, especially images containing significant objects, the objects are generally located in the center of the image, and the human eye is more sensitive to the part close to the center of the image. Therefore, the application can sort the image regions according to the position distance between each image region and the image center of the image to be matched.

[0087] To accurately calculate the similarity of the two image regions, in an embodiment of the image matching method of the application, referring to Figure 3 the step S102 can further include the following content:

[0088] Step S301: sequentially determine whether the two image regions of the two images to be matched have the same visual attributes according to the importance order.

[0089] Step S302: determine the similarity of the two image regions according to the number of the same visual attributes possessed by the two image regions and the spatial descriptors of the two image regions.

[0090] Optionally, after the importance of the multiple image regions of the two images to be matched is sorted, the application can compare the features of the two image regions in the importance order, for example, compare the features of the image region with the highest importance in one image to be matched and the image region with the highest importance in the other image to be matched, and determine the similarity of the two image regions according to the feature comparison result.

[0091] Optionally, the application can calculate the similarity of the two image regions based on the same visual attributes possessed by the two image regions and the spatial descriptors of the two image regions.

[0092] Optionally, if two image regions (for example, a certain region of a query image and a certain region of an image set) of two images to be matched contain the same visual attributes, the two image regions are a pair of matched regions (RP). The similarity of the two image regions can be calculated using the following formula:

[0093]

[0094] In the formula, m represents the number of matched sub-region pairs in the two image regions, SD and SR represent the spatial descriptors of the four visual attributes in a pair of sub-region pairs, and the higher the score of MS, the more similar the two image regions.

[0095] In order to effectively improve the image retrieval accuracy and matching efficiency, an embodiment of an image matching device for implementing all or part of the image matching method is provided, as shown in Figure 4 , and the image matching device specifically includes the following contents:

[0096] An image region division module 10 is configured to determine image feature points of two images to be matched, perform coordinate clustering based on the image feature points, and divide the two images to be matched into a plurality of image regions based on the coordinate clustering results.

[0097] A salient region determination module 20 is configured to sort a plurality of image regions of each image to be matched according to importance, perform feature comparison on two image regions of the two images to be matched in sequence according to the importance order, and determine the similarity of the two image regions.

[0098] An image matching module 30 is configured to determine whether the two images to be matched are matched based on the similarity of the two image regions.

[0099] As can be seen from the above description, the image matching device provided by the embodiment of the present application can divide the entire image into “salient” and “non-salient” parts by dividing the image regions, compare the image features of the salient regions, thereby greatly reducing the number of image features, and improve the image retrieval accuracy and image retrieval matching efficiency.

[0100] In order to accurately extract the image feature points of the images to be matched, in an embodiment of the image matching device of the present application, as shown in Figure 5 , the image region division module 10 includes:

[0101] A feature point determination unit 11 is configured to find extreme points of the images to be matched in different scale spaces based on a preset scale-invariant feature transformation algorithm, and obtain the image feature points of the images to be matched.

[0102] In order to accurately divide the to-be-matched image into multiple image regions, in an embodiment of the image matching device of the present application, referring to Figure 6 , the image region dividing module 10 further comprises:

[0103] a coordinate clustering unit 12 configured to perform coordinate clustering on the coordinates of the image feature points according to a preset mean shift clustering algorithm.

[0104] an image dividing unit 13 configured to determine multiple image regions according to the result of the coordinate clustering.

[0105] In order to accurately determine the representative regions among the multiple image regions, in an embodiment of the image matching device of the present application, referring to Figure 7 , the significant region determining module 20 comprises:

[0106] a quantity sorting unit 21 configured to sort the importance of each image region according to the number of image feature points contained in each divided image region.

[0107] In order to accurately determine the representative regions among the multiple image regions, in an embodiment of the image matching device of the present application, referring to Figure 8 , the significant region determining module 20 further comprises:

[0108] a distance sorting unit 22 configured to sort the importance of each image region according to the position distance between each divided image region and the image center of the to-be-matched image.

[0109] In order to accurately calculate the similarity between two image regions, in an embodiment of the image matching device of the present application, referring to Figure 9 , the significant region determining module 20 further comprises:

[0110] a same attribute judging unit 23 configured to determine whether the two image regions of the two to-be-matched images have the same visual attribute in sequence according to the importance order.

[0111] a similarity calculating unit 24 configured to determine the similarity between the two image regions according to the number of the same visual attributes possessed by the two image regions and the spatial descriptor of the two image regions.

[0112] From the hardware aspect, in order to effectively improve the image retrieval accuracy and matching efficiency, the present application provides an embodiment of an electronic device for implementing all or part of the contents of the image matching method, which specifically comprises the following contents:

[0113] A processor, a memory, a communications interface, and a bus; wherein the processor, the memory, the communications interface complete the communication among each other through the bus; the communications interface is used for realizing the information transmission between the image matching device and the core service system, the user terminal, and the related database and other related devices; the logic controller can be a desktop computer, a tablet computer, a mobile terminal, and the like, and the embodiment is not limited thereto. In the embodiment, the logic controller can be implemented by referring to the embodiments of the image matching method and the embodiments of the image matching device, the contents of which are incorporated herein, and the repeated parts are not described herein.

[0114] It can be understood that the user terminal can include a smart phone, a tablet electronic device, a network set-top box, a portable computer, a desktop computer, a personal digital assistant (PDA), a vehicle-mounted device, a smart wearable device, and the like. The smart wearable device can include smart glasses, a smart watch, a smart bracelet, and the like.

[0115] In actual application, part of the image matching method can be executed on the electronic device as described above, or all operations can be completed in the client device. Specifically, the selection can be made according to the processing capability of the client device and the limitation of the user use scenario, and the like. The present application is not limited thereto. If all operations are completed in the client device, the client device can further include a processor.

[0116] The above-mentioned client device can have a communication module (i.e., a communication unit) and can be communicatively connected with a remote server to realize the data transmission with the server. The server can include a server of the task scheduling center side, and the server of the intermediate platform can also be included in other implementation scenarios, for example, the server of the third-party server platform which is communicatively connected with the server of the task scheduling center. The server can include a single computer device, a server cluster composed of multiple servers, or a distributed server structure.

[0117] Figure 10 A schematic block diagram of the system configuration of the electronic device 9600 of the embodiment of the present application is shown in FIG. 9. As shown in FIG. 9, the electronic device 9600 can include a central processor 9100 and a memory 9140; the memory 9140 is coupled to the central processor 9100. It is worth noting that the structure shown in FIG. 9 is exemplary; other types of structures can also be used to supplement or replace the structure to realize the telecommunication function or other functions. Figure 10 Figure 10 The structure shown in FIG. 9 is exemplary; other types of structures can also be used to supplement or replace the structure to realize the telecommunication function or other functions.

[0118] ​In one embodiment, the image matching method function may be integrated into the central processing unit 9100. The central processing unit 9100 may be configured to perform the following control:

[0119] Step S101: determining image feature points of two images to be matched, performing coordinate clustering according to the image feature points, and dividing the images to be matched into multiple image regions according to the coordinate clustering results.

[0120] Step S102: sorting the importance of the multiple image regions of each image to be matched, and performing feature comparison on the two image regions of the two images to be matched in order according to the importance order to determine the similarity between the two image regions.

[0121] Step S103: determining whether the two images to be matched match according to the similarity of the two image regions.

[0122] From the above description, it can be seen that the electronic device provided in the embodiment of the present application divides the image area so that the entire image is divided into two parts, "salient" and "non-salient", and compares the image features of the significant area, thereby greatly reducing the number of image features, which is conducive to improving image retrieval accuracy and image retrieval matching efficiency.

[0123] In another embodiment, the image matching device may be configured separately from the central processing unit 9100. For example, the image matching device may be configured as a chip connected to the central processing unit 9100, and the image matching method function may be implemented under the control of the central processing unit.

[0124] like Figure 10 As shown, the electronic device 9600 may further include: a communication module 9110, an input unit 9120, an audio processor 9130, a display 9160, and a power supply 9170. It is worth noting that the electronic device 9600 does not necessarily have to include Figure 10 In addition, the electronic device 9600 may also include all components shown in Figure 10 For components not shown, reference may be made to the prior art.

[0125] like Figure 10 As shown, the central processing unit 9100 is sometimes also referred to as a controller or operation control, and may include a microprocessor or other processor device and / or logic device. The central processing unit 9100 receives input and controls the operation of various components of the electronic device 9600.

[0126] The memory 9140, for example, can be one or more of a buffer, a flash memory, a hard drive, a removable media, a volatile memory, a non-volatile memory, or other suitable device. The above-mentioned information related to failure can be stored, and in addition, a program for executing the information related to failure can be stored. The central processing unit 9100 can execute the program stored in the memory 9140 to achieve information storage or processing, and the like.

[0127] The input unit 9120 provides input to the central processing unit 9100. The input unit 9120 is, for example, a key or a touch input device. The power supply 9170 is used to supply power to the electronic device 9600. The display 9160 is used to display display objects such as images and characters. The display can be, for example, an LCD display, but is not limited thereto.

[0128] The memory 9140 can be a solid state memory such as a read only memory (ROM), a random access memory (RAM), a SIM card, and the like. It can also be a memory that retains information even when power is off, can be selectively erased, and is provided with more data, and examples of such a memory are sometimes referred to as an EPROM, and the like. The memory 9140 can also be some other type of device. The memory 9140 includes a buffer memory 9141 (sometimes referred to as a buffer). The memory 9140 can include an application / function storage section 9142 for storing application programs and function programs or for storing a flow for executing operations of the electronic device 9600 by the central processing unit 9100.

[0129] The memory 9140 can also include a data storage section 9143 for storing data such as contacts, digital data, pictures, sounds, and / or any other data used by the electronic device. A driver storage section 9144 of the memory 9140 can include various drivers of the electronic device for a communication function and / or for executing other functions of the electronic device such as a messaging application, an address book application, and the like.

[0130] The communication module 9110 is a transmitter / receiver 9110 that transmits and receives signals via an antenna 9111. The communication module (transmitter / receiver) 9110 is coupled to the central processing unit 9100 to provide input signals and receive output signals, which can be the same as in the case of a conventional mobile communication terminal.

[0131] Based on different communication technologies, multiple communication modules 9110, such as a cellular network module, a Bluetooth module, and / or a wireless local area network module, etc., can be provided in the same electronic device. The communication module (transmitter / receiver) 9110 is also coupled to a speaker 9131 and a microphone 9132 via an audio processor 9130 to provide audio output via the speaker 9131 and to receive audio input from the microphone 9132, thereby enabling typical telecommunication functions. The audio processor 9130 can include any suitable buffers, decoders, amplifiers, etc. In addition, the audio processor 9130 is coupled to the central processor 9100, thereby enabling recording on the local device via the microphone 9132 and enabling playing of stored sounds on the local device via the speaker 9131.

[0132] The embodiments of the present application further provide a computer readable storage medium capable of implementing all steps of the image matching method in the above-mentioned embodiments in which the execution subject is a server or a client. The computer program is stored on the computer readable storage medium and is executed by a processor to implement all steps of the image matching method in the above-mentioned embodiments in which the execution subject is a server or a client. For example, the computer program implements the following steps when executed by the processor:

[0133] Step S101: determining image feature points of two images to be matched, performing coordinate clustering according to the image feature points, and dividing the images to be matched into multiple image regions according to the coordinate clustering result.

[0134] Step S102: performing importance sorting on the multiple image regions of each image to be matched, performing feature comparison on two image regions of the two images to be matched in sequence according to the importance order, and determining the similarity of the two image regions.

[0135] Step S103: determining whether the two images to be matched are matched according to the similarity of the two image regions.

[0136] As can be seen from the above description, the computer readable storage medium provided by the embodiments of the present application divides the entire image into two parts of “significant” and “insignificant” through image region division, compares the image features of the significant regions, thereby greatly reducing the number of image features, and is conducive to improving the image retrieval accuracy and image retrieval matching efficiency.

[0137] The embodiments of the present application further provide a computer program product capable of implementing all steps of the image matching method in the above-mentioned embodiments in which the execution subject is a server or a client. The computer program / instruction is executed by a processor to implement the steps of the image matching method. For example, the computer program / instruction implements the following steps:

[0138] Step S101: determining image feature points of two images to be matched, clustering coordinates according to the image feature points, and dividing the images to be matched into a plurality of image regions according to the clustering results of the coordinates.

[0139] Step S102: sorting the image regions of each image to be matched according to importance, and performing feature comparison on two image regions of the two images to be matched according to the importance order, to determine the similarity of the two image regions.

[0140] Step S103: determining whether the two images to be matched are matched according to the similarity of the two image regions.

[0141] From the above description, it can be seen that the computer program product provided by the embodiment of the present application divides the image into two parts of "significant" and "insignificant" through the division of the image region, compares the image features of the significant region, thereby greatly reducing the number of image features, and is beneficial to improving the image retrieval precision and image retrieval matching efficiency.

[0142] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, device, or computer program product. Therefore, the present application can adopt a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer usable storage media (including but not limited to disk memory, CD-ROM, optical memory, etc.) containing computer usable program codes.

[0143] The present application is described with reference to flowcharts and / or block diagrams of the method, device (apparatus), and computer program product according to the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of the flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device that implements the flowcharts and / or block diagrams. Figure 1 The function specified in one flow or multiple flows and / or blocks. Figure 1 The device that implements the function specified in one flow or multiple flows and / or blocks.

[0144] These computer program instructions can also be stored in a computer readable memory capable of guiding the computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce a product including instruction devices, which implement the flowcharts and / or block diagrams. Figure 1 The function specified in one flow or multiple flows and / or blocks. Figure 1the function specified in the one or more blocks.

[0145] These computer program instructions can also be loaded into computer or other programmable data processing devices, so that a series of operational steps are generated to realize the computer-implemented process, and the instructions executed on the computer or other programmable devices provide a process for implementing the flowchart Figure 1 the flowchart or flowcharts and / or a block Figure 1 the steps of the function specified in the one or more blocks.

[0146] The principles and implementation manners of the present application are described in the specific embodiments. The above embodiment descriptions are only used to help understand the method and core idea of the present application; meanwhile, for those skilled in the art, according to the idea of the present application, the specific implementation manners and application ranges will be changed, and the above descriptions should not be understood as limitations on the present application.

Claims

1. An image matching method, characterized in that: The method comprises: Determining image feature points of two to-be-matched images, performing coordinate clustering based on the image feature points, and dividing the to-be-matched images into a plurality of image regions based on the coordinate clustering results; sorting the importance of the multiple image regions of each to-be-matched image, and performing feature comparison on the two image regions of the two to-be-matched images in order of importance to determine the similarity between the two image regions; Determining whether the two images to be matched match according to the similarity of the two image regions; The step of comparing features of the two image regions of the two images to be matched in order of importance to determine the similarity of the two image regions includes: determining, in order of importance, whether the two image regions of the two to-be-matched images have the same visual attributes; The similarity between the two image regions is determined according to the number of the same visual attributes of the two image regions and the spatial descriptors of the two image regions using the following formula: Where m represents the number of matching sub-region pairs in the two image regions, SD and SR represent the spatial descriptors of the four visual attributes in a pair of sub-region matching pairs, and the higher the MS score, the more similar the two image regions are.

2. The image matching method according to claim 1, wherein: Determining the image feature points of the two images to be matched includes: The extreme points of the image to be matched in different scale spaces are found according to a preset scale-invariant feature conversion algorithm to obtain image feature points of the image to be matched.

3. The image matching method according to claim 1, wherein: The performing coordinate clustering according to the image feature points and dividing the image to be matched into a plurality of image regions according to the coordinate clustering result includes: performing coordinate clustering on the coordinates of the image feature points according to a preset mean shift clustering algorithm; A plurality of image regions are determined according to the coordinate clustering result.

4. The image matching method according to claim 1, wherein: The step of ranking the plurality of image regions of each image to be matched by importance includes: The importance of each image region is sorted according to the number of image feature points contained in each divided image region.

5. The image matching method according to claim 1, wherein: The ordering of importance of the multiple image regions of each image to be matched further includes: The importance of each image region is sorted according to the position distance between each divided image region and the image center of the image to be matched.

6. An image matching device, characterized in that: include: An image region division module is used to determine image feature points of two images to be matched, perform coordinate clustering based on the image feature points, and divide the images to be matched into multiple image regions based on the coordinate clustering results; a salient region determination module, configured to sort the importance of the multiple image regions of each to-be-matched image, and perform feature comparison on the two image regions of the two to-be-matched images in order of importance, to determine the similarity between the two image regions; An image matching module, configured to determine whether the two images to be matched match based on the similarity of the two image regions; The salient region determination module is specifically configured to determine, in order of importance, whether the two image regions of the two to-be-matched images have the same visual attributes; The similarity between the two image regions is determined according to the number of the same visual attributes of the two image regions and the spatial descriptors of the two image regions using the following formula: Where m represents the number of matching sub-region pairs in the two image regions, SD and SR represent the spatial descriptors of the four visual attributes in a pair of sub-region matching pairs, and the higher the MS score, the more similar the two image regions are.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the steps of the image matching method according to any one of claims 1 to 5 are implemented.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the image matching method according to any one of claims 1 to 5 are implemented.

9. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instruction is executed by a processor, the steps of the image matching method according to any one of claims 1 to 5 are implemented.

Images