Image matching method, device, intelligent device and storage medium

Abstract

The embodiments of the present application disclose an image matching method, apparatus, intelligent device and computer-readable storage medium. The method includes: obtaining a template image corresponding to the image to be matched, performing feature extraction on the template image and the image to be matched respectively, obtaining gradient information of M first key feature points of the template image and gradient information of N second key feature points of the image to be matched, calculating similarity information of P sub-regions in the image to be matched based on the gradient information of the M first key feature points and the gradient information of the second key feature points included in each sub-region of the image to be matched, and determining the target sub-region from the P sub-regions based on the similarity information of the P sub-regions. It can be seen that by jointly calculating the similarity information of the P sub-regions by the gradient value and the gradient direction, interference points (such as feature points with the same gradient value but different gradient directions) can be further filtered out, thereby improving the accuracy of image matching.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to an image matching method, apparatus, intelligent device and computer-readable storage medium. Background Art

[0002] With the continuous development of computer technology, image matching technology has been widely applied in fields such as industrial automation, object detection, and video detection. For example, in industrial production, machine vision inspection equipment can quickly detect defective products (such as missing components and whether the product appearance meets standards), reducing labor costs and improving inspection efficiency. However, in practice, image matching technology typically relies on contour matching, resulting in low accuracy. Summary of the Invention

[0003] Embodiments of the present invention provide an image matching method, apparatus, intelligent device, and computer-readable storage medium, which can improve the accuracy of image matching.

[0004] In one aspect, an embodiment of the present application provides an image matching method, comprising:

[0005] Obtaining a to-be-matched image and a template image corresponding to the to-be-matched image;

[0006] Performing feature extraction on the template image to obtain gradient information of M first key feature points of the template image, where the gradient information of the first key feature points includes a gradient value and a gradient direction of the first key feature points, where M is a positive integer;

[0007] Performing feature extraction on the image to be matched to obtain gradient information of N second key feature points of the image to be matched, where the gradient information of the second key feature points includes a gradient value and a gradient direction set of the second key feature points, where N is a positive integer;

[0008] Calculate similarity information of P sub-regions in the to-be-matched image based on the gradient information of the M first key feature points and the gradient information of the second key feature points included in each sub-region of the to-be-matched image, where P is a positive integer;

[0009] Determining a target subregion from the P subregions according to the similarity information of the P subregions;

[0010] Among them, the similarity of each sub-region in the to-be-matched image is obtained based on the target feature points in each sub-region in the to-be-matched image; the target feature points belong to the N second key feature points, and the gradient direction set of the target feature points contains the gradient direction of the first key feature point corresponding to the target feature point.

[0011] In one aspect, an embodiment of the present application provides an image matching device, comprising:

[0012] An acquisition unit, configured to acquire a to-be-matched image and a template image corresponding to the to-be-matched image;

[0013] a processing unit, configured to perform feature extraction on the template image to obtain gradient information of M first key feature points of the template image, wherein the gradient information of the first key feature points includes a gradient value and a gradient direction of the first key feature points, and M is a positive integer;

[0014] Performing feature extraction on the image to be matched to obtain gradient information of N second key feature points of the image to be matched, where the gradient information of the second key feature points includes a gradient value and a gradient direction set of the second key feature points, where N is a positive integer;

[0015] Calculate similarity information of P sub-regions in the to-be-matched image based on the gradient information of the M first key feature points and the gradient information of the second key feature points included in each sub-region of the to-be-matched image, where P is a positive integer;

[0016] Determining a target subregion from the P subregions according to the similarity information of the P subregions;

[0017] Among them, the similarity of each sub-region in the to-be-matched image is obtained based on the target feature points in each sub-region in the to-be-matched image; the target feature points belong to the N second key feature points, and the gradient direction set of the target feature points contains the gradient direction of the first key feature point corresponding to the target feature point.

[0018] Accordingly, an embodiment of the present application also provides an intelligent device, including a processor, a memory and a communication interface, wherein the processor, the memory and the communication interface are interconnected, wherein the memory is used to store a computer program, the computer program includes program instructions, and the processor is configured to call the program instructions to execute the above-mentioned image matching method.

[0019] Accordingly, the present application provides a computer-readable storage medium, which stores one or more instructions, and the one or more instructions are suitable for being loaded by a processor and executing the above-mentioned image matching method.

[0020] Accordingly, the present application provides a computer program product or computer program, which includes computer instructions, which are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the above-mentioned image matching method.

[0021] In an embodiment of the present application, a template image corresponding to the image to be matched is obtained, and feature extraction is performed on each of the template image and the image to be matched to obtain gradient information of M first key feature points of the template image and gradient information of N second key feature points of the image to be matched. Based on the gradient information of the M first key feature points and the gradient information of the second key feature points included in each sub-region of the image to be matched, similarity information of P sub-regions in the image to be matched is calculated, and based on the similarity information of the P sub-regions, a target sub-region is determined from among the P sub-regions. It can be seen that, compared to determining the similarity information of P sub-regions using a single feature (such as a gradient value), calculating the similarity information of P sub-regions using both the gradient value and the gradient direction can further filter out interference points (such as feature points with the same gradient value but different gradient directions), thereby improving the accuracy of image matching. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0023] Figure 1 An application scenario diagram of image matching provided in an embodiment of the present application;

[0024] Figure 2 A flowchart of an image matching method provided in an embodiment of the present application;

[0025] Figure 3 A schematic diagram of image matching provided in an embodiment of the present application;

[0026] Figure 4 A flowchart of another image matching method provided in an embodiment of the present application;

[0027] Figure 5a A schematic diagram of a template diagram provided in an embodiment of the present application;

[0028] Figure 5b A schematic diagram of a mask pattern provided in an embodiment of the present application;

[0029] Figure 5c A schematic diagram of adjacent feature points provided in an embodiment of the present application;

[0030] Figure 5d A schematic diagram of determining a gradient direction set provided in an embodiment of the present application;

[0031] Figure 5eA matching result diagram provided in an embodiment of the present application;

[0032] Figure 5f An architectural diagram of an image matching system provided in an embodiment of the present application;

[0033] Figure 6 A schematic diagram of the structure of an image matching device provided in an embodiment of the present application;

[0034] Figure 7 A schematic diagram of the structure of a smart device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0035] The technical solutions in the embodiments of the present invention will be described clearly and completely below with reference to the accompanying drawings in the embodiments of the present invention.

[0036] The embodiments of the present application involve artificial intelligence (AI) and machine learning (ML). By combining AI and ML, it is possible to mine and analyze features (such as gradient values ​​and gradient directions) in the template image and the image to be matched, enabling intelligent devices to perform feature matching based on the features of the template image and the image to be matched, and then determine the target sub-region in the image to be matched that matches the template image. AI is the theory, method, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use knowledge to achieve optimal results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new type of intelligent machine that can respond in a manner similar to human intelligence. Artificial intelligence is the study of the design principles and implementation methods of various intelligent machines, giving them the capabilities of perception, reasoning, and decision-making.

[0037] AI technology is a comprehensive discipline encompassing a wide range of fields, encompassing both hardware and software technologies. Basic AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, processing technologies for large applications, operating / interaction systems, and mechatronics. AI software technologies primarily encompass computer vision, speech processing, natural language processing, and machine learning / deep learning. Computer vision is the study of how to enable machines to "see." Specifically, it refers to machine vision techniques that use cameras and computers to replace the human eye in identifying, detecting, and measuring objects, and further image processing to create images more suitable for human observation or transmission to instruments. As a scientific discipline, computer vision studies related theories and technologies, attempting to build AI systems capable of extracting information from images or multidimensional data. These techniques typically include image processing, video processing, video semantic understanding, and video content / behavior recognition. The present application embodiments primarily relate to image matching within computer vision, specifically identifying target subregions within an image to be matched based on the features of a template image.

[0038] Machine learning is a multi-disciplinary interdisciplinary subject that involves probability theory, statistics, approximation theory, convex analysis, algorithmic complexity theory and other disciplines. It specializes in studying how computers simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their own performance. Machine learning is the core of AI and the fundamental way to make computers intelligent. Its applications are spread across all areas of artificial intelligence. Machine learning / deep learning usually includes artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and self-learning. This application mainly involves using sample data to train the initial model to obtain a trained image matching model.

[0039] See Figure 1 , Figure 1 This is an application scenario diagram of image matching provided in the embodiment of the present application. Figure 1 As shown, the image matching application scenario includes a capture device 101 and a smart device 102. Capture device 101 is an image acquisition device that can be used to capture a template image and an image to be matched (the image to be matched is obtained by photographing the object to be detected or by other means). Capture device 101 may include, but is not limited to, cameras, smartphones (such as Android phones and iOS phones), tablet computers, mobile internet devices (MIDs), and other devices with a camera function.

[0040] Smart device 102 refers to a backend device capable of providing image matching services for the image to be matched captured by acquisition device 101 (i.e., determining target subregions in the image to be matched sent from acquisition device 101 that match the template image and outputting matching results). Smart device 102 can be a standalone physical server or computer (a computer is often equipped with a display device, which can be a monitor, display screen, touch screen, etc., and a touch screen can be a touch screen, touch panel, etc., but this is not limited in this embodiment of the present invention). It can also be a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. In addition, multiple smart devices can be organized into a blockchain network, with each smart device acting as a node in the blockchain network. The acquisition device 101 and smart device 102 can be connected directly or indirectly via wired or wireless communication, which is not limited in this application.

[0041] It should be noted that Figure 1 The number of acquisition devices and smart devices in the image matching application scenario shown is only an example. For example, the number of acquisition devices and smart devices can be multiple, and this application does not limit the number of acquisition devices and smart devices.

[0042] Optionally, the template image may be sent to the smart device 102 after the acquisition device 101 captures an image of the target object. After obtaining the template image sent by the acquisition device 101, the smart device 102 may process the template image (e.g., sharpen the image) to make the features in the processed template image more prominent. The application scenario of image matching may also only include the acquisition device 101 equipped with an image matching device. After acquiring the image to be matched and the template image, the acquisition device 101 processes the acquired image to be matched and the template image using the image matching device, and returns the matching result (e.g., the target sub-region that matches the template image) to the user (e.g., displaying the result on a display screen or sending the result to the target device).

[0043] Figure 1In the application scenario of image matching shown in the figure, the image matching process mainly includes the following steps: (1) The smart device obtains the template image corresponding to the image to be matched. The template image is created or indicated by the user. The template image is used to indicate the object (such as parts, animals, plants, etc.) that needs to be searched in the image to be matched; for example, the image to be matched is a picture containing 10 parts, and the template image is a picture of one or more of the 10 parts. (2) The smart device extracts features from the template image (such as calculating the initial gradient value and initial gradient direction of the template image based on the grayscale of the template image) to obtain the gradient information of the M first key feature points of the template image. The gradient information of the first key feature point includes the gradient value and gradient direction of the first key feature point, and M is a positive integer. (3) The smart device extracts features from the image to be matched (such as calculating the initial gradient value and initial gradient direction of the image to be matched based on the grayscale of the image to be matched) to obtain the gradient information of the N second key feature points of the image to be matched. The gradient information of the second key feature point includes the gradient value and gradient direction set of the second key feature point, and N is a positive integer. Step (3) may also be performed before step (2) or simultaneously with step (2), and this application does not impose any restrictions on this. (4) The intelligent device calculates the similarity information (the similarity information includes the similarity between each sub-region and the template image) of P sub-regions in the image to be matched (for example, P sub-regions are selected from the image to be matched by sliding a window) based on the gradient information of the M first key feature points and the gradient information of the second key feature points included in each sub-region of the image to be matched, where P is a positive integer; that is, the similarity information of each sub-region is calculated based on the gradient information of the M first key feature points and the feature information of the second key feature points included in the sub-region. (5) The intelligent device determines the target sub-region from the P sub-regions based on the similarity information of the P sub-regions (for example, the sub-region with the largest similarity among the P sub-regions is determined as the target sub-region).

[0044] In an embodiment of the present application, a template image corresponding to the image to be matched is obtained, and feature extraction is performed on each of the template image and the image to be matched to obtain gradient information of M first key feature points of the template image and gradient information of N second key feature points of the image to be matched. Based on the gradient information of the M first key feature points and the gradient information of the second key feature points included in each sub-region of the image to be matched, similarity information of P sub-regions in the image to be matched is calculated, and based on the similarity information of the P sub-regions, a target sub-region is determined from among the P sub-regions. It can be seen that, compared to determining the similarity information of P sub-regions using a single feature (such as a gradient value), calculating the similarity information of P sub-regions using both the gradient value and the gradient direction can further filter out interference points (such as feature points with the same gradient value but different gradient directions), thereby improving the accuracy of image matching.

[0045] See also Figure 2 , Figure 2This is a flowchart of an image matching method provided in an embodiment of the present application. The method is used to process an image to be matched that is captured or otherwise obtained. The method can be executed by a smart device, which can be a terminal device or a server. The method in this embodiment of the present application may include the following steps.

[0046] S201: Obtain a to-be-matched image and a template image corresponding to the to-be-matched image. The to-be-matched image refers to an image that requires matching processing (i.e., searching and confirming the area in the to-be-matched image that matches the template image corresponding to the to-be-matched image). In one embodiment, the to-be-matched image is captured by an acquisition device (such as a camera) or retrieved from a network, data storage device, etc. The template image is an image that indicates the object (such as a part, animal, plant, etc.) to be searched for in the to-be-matched image. For example, the to-be-matched image is an image containing 10 parts, and the template image is an image of one or more of the 10 parts.

[0047] S202: Extract features from the template image to obtain gradient information of M first key feature points of the template image. The gradient information of the first key feature points includes the gradient value and gradient direction of the first key feature points, where M is a positive integer.

[0048] In one embodiment, feature extraction from a template image corresponding to a to-be-matched image includes performing edge detection processing on the template image using an edge detection algorithm (e.g., calculating a gradient map corresponding to the template image using the edge detection algorithm, and obtaining the original gradient value and original gradient direction of each original feature point in the template image through the gradient map), and filtering and calibrating the edge detection results to obtain gradient information for M first key feature points in the template image. The gradient information of the M first key feature points is used to indicate features of a predicted edge region in the template image. The edge detection algorithm may include, but is not limited to, a local difference (Roberts) operator, a pixel average (Prewitt) operator, and a weighted average (Sobel) operator.

[0049] Specifically, after obtaining the original gradient value and original gradient direction of each original feature point in the template image, the smart device filters the original feature points using a first effective threshold (e.g., retaining original feature points whose original gradient values ​​are greater than the first effective threshold), thereby obtaining M first key feature points. Furthermore, the smart device normalizes the gradient values ​​of the M filtered first key feature points and calibrates the gradient directions of the M first key feature points, thereby obtaining gradient information for the M first key feature points.

[0050] S203: Perform feature extraction on the image to be matched to obtain gradient information for N second key feature points of the image to be matched. The gradient information for the second key feature points includes a gradient value and a gradient direction set for the second key feature points, where N is a positive integer. In one embodiment, the object in the template image is included in the image to be matched, and in this case, M is less than or equal to N.

[0051] In one embodiment, feature extraction from the image to be matched includes performing edge detection processing on objects in the image to be matched using an edge detection algorithm (e.g., calculating a gradient map corresponding to the image to be matched using the edge detection algorithm, and obtaining original gradient values ​​and original gradient directions of each original feature point in the image to be matched using the gradient map), thereby obtaining gradient information of N second key feature points of the object. The gradient information of the N second key feature points is used to indicate edge features of one or more objects in the image to be matched.

[0052] Specifically, after obtaining the original gradient value and original gradient direction of each original feature point in the image to be matched, the smart device filters the original feature points using a second effective threshold (e.g., retaining original feature points whose original gradient values ​​are greater than the second effective threshold), thereby obtaining N second key feature points. Furthermore, the smart device normalizes the gradient values ​​of the N filtered second key feature points and, based on the original gradient directions of the N second key feature points, determines a set of gradient directions for the N second key feature points.

[0053] S204: Calculate similarity information for the P sub-regions in the image to be matched based on the gradient information of the M first key feature points and the gradient information of the second key feature points included in each sub-region of the image to be matched. The similarity of each sub-region in the image to be matched is obtained based on target feature points in each sub-region in the image to be matched (e.g., counting the number of target feature points in each sub-region in the image to be matched); the target feature point belongs to the N second key feature points, and the gradient direction set of the target feature point includes the gradient direction of the first key feature point corresponding to the target feature point.

[0054] In one embodiment, each sub-region of the image to be matched is obtained by sliding a window in the image to be matched. The size of the sliding window is the same as that of the template image. Specifically, the starting position of the sliding window is obtained, and the sliding window is slid in the image to be matched according to the sliding rule (such as starting from the upper left corner of the image to be matched, in order from left to right and from top to bottom), and the step length of each sliding is a pixels, where a is a positive integer; for example, assuming that the step length of each sliding is 1 pixel, the image to be matched is an image with a length and width of 999 pixels, and the template image is an image with a length and width of 100 pixels, then the sliding window is used to slide in the image to be matched to obtain (999-100+1) (999-100+1)=810,000 sub-regions.

[0055] The specific implementation method for the smart device to calculate the similarity information of each sub-region is as follows: detect the cth second key feature point contained in the bth sub-region, and whether there is a corresponding first key feature point at the corresponding position in the template image; if there is a first key feature point corresponding to the cth second key feature point at the corresponding position in the template image, then detect whether the gradient direction set of the cth second key feature point contains the gradient direction of the first key feature point corresponding to the cth second key feature point; if the gradient direction set of the cth second key feature point contains the gradient direction of the first key feature point corresponding to the cth second key feature point, then determine the cth second key feature point as the target feature point; b and c are positive integers.

[0056] Figure 3 This is a schematic diagram of image matching provided in the embodiment of the present application. Figure 3 As shown, suppose the size of the gradient image corresponding to the image to be matched is N x N (pixels), the size of the gradient image corresponding to the template image is M x M (pixels), and the arbitrary point position I(i,j) is a sub-region with a length and width of M x M (pixels). The coordinates of the upper left corner in the gradient map of the image to be matched, T(x,y) is used to represent the coordinate position of the pixel point in the gradient map of the template image, then the sub-region of the I(i,j) position Similarity with the template graph It can be expressed as:

[0057]

[0058] Among them, the function Indicates that the template image and the feature points in the image to be matched are compared. If The gradient value of the feature point is greater than the effective threshold (such as the gradient value is greater than 0), and the gradient direction of the feature point is included in In the gradient set of feature points, ,otherwise .

[0059] The target feature points contained in each sub-region are determined in the above manner, and the number of target feature points contained in each sub-region is counted to obtain the similarity information of each sub-region. The similarity of each sub-region is proportional to the number of target feature points.

[0060] Optionally, the P sub-regions are sub-regions within the limited matching region in the image to be matched. For example, if the region of the template image in the image to be matched is known, the region is set as the limited matching region, and the sliding window is traversed within the limited matching region to obtain the P sub-regions.

[0061] S205: Determine a target sub-region from the P sub-regions according to the similarity information of the P sub-regions.

[0062] In one embodiment, the sub-region with the highest similarity among the P sub-regions is determined as the target sub-region (ie, the matching region that matches the template image).

[0063] In an embodiment of the present application, a template image corresponding to the image to be matched is obtained, and feature extraction is performed on each of the template image and the image to be matched to obtain gradient information of M first key feature points of the template image and gradient information of N second key feature points of the image to be matched. Based on the gradient information of the M first key feature points and the gradient information of the second key feature points included in each sub-region of the image to be matched, similarity information of P sub-regions in the image to be matched is calculated, and based on the similarity information of the P sub-regions, a target sub-region is determined from among the P sub-regions. It can be seen that, compared to determining the similarity information of P sub-regions using a single feature (such as a gradient value), calculating the similarity information of P sub-regions using both the gradient value and the gradient direction can further filter out interference points (such as feature points with the same gradient value but different gradient directions), thereby improving the accuracy of image matching.

[0064] See also Figure 4 , Figure 4 A flowchart of another image matching method provided in an embodiment of the present application. The method is used to process an image to be matched, obtained by photographing or otherwise, and can be executed by a smart device, which can specifically be a terminal device or a server. The method in the embodiment of the present application may include the following steps.

[0065] S401: Obtain a to-be-matched image and a template image corresponding to the to-be-matched image.

[0066] The specific implementation of step S401 can be found in Figure 2 The implementation of step S201 will not be repeated here.

[0067] S402: Extract features from the template image to obtain gradient information of K first original feature points of the template image; K is a positive integer.

[0068] In one embodiment, a mask image that matches the template image is obtained. The mask image is a binary image corresponding to the template image, where a white area represents a foreground and a black area represents a background. Figure 5a A schematic diagram of a template diagram provided in an embodiment of the present application, such as Figure 5a As shown, the gray part is the part and the remaining stripes are the background. Figure 5b A schematic diagram of a mask diagram provided in an embodiment of the present application is shown as follows: Figure 5b As shown, the white area is Figure 5a The part in the foreground, the black area is Figure 5a The smart device determines the area in the template image that matches the foreground area as the valid area of ​​the template image, and performs feature extraction on the valid area of ​​the template image to obtain the gradient information of the K first original feature points of the template image.

[0069] Specifically, feature extraction from the template image includes performing edge detection on the template image using an edge detection algorithm. Specifically, the edge detection algorithm calculates a gradient map corresponding to a valid area of ​​the template image. The gradient map includes K first original feature points, and gradient information (original gradient values ​​and original gradient directions) of the K first original feature points is obtained from the gradient map. The edge detection algorithm may include, but is not limited to, a local difference (Roberts) operator, a pixel average (Prewitt) operator, and a weighted average (Sobel) operator.

[0070] S403: Determine M first key feature points from the K first original feature points according to a first effective threshold.

[0071] In one embodiment, K first original feature points are filtered using a first effective threshold to obtain M first key feature points, each of which has a feature value greater than the first effective threshold. For example, assuming the first effective threshold is 5, the feature value of original feature point 1 is 3, the feature value of original feature point 2 is 7, and the feature value of original feature point 3 is 12; then, original feature points 1-3 are filtered using the first effective threshold to obtain two first key feature points (i.e., original feature point 2 and original feature point 3). Filtering the first original feature points using the first effective threshold can filter out noise feature points in the first original feature points, thereby enhancing the robustness of the image matching results.

[0072] In another embodiment, if the gradient value of the rth first original feature point among the K first original feature points is greater than the first effective threshold, and the gradient value of the rth first original feature point is greater than or equal to the gradient value of each first original feature point in the set of adjacent feature points of the rth original feature point, then the rth original feature point is determined as the first key feature point of the template image, where r is a positive integer and is less than or equal to K. Specifically, after screening the K first original feature points using the first effective threshold, Y candidate feature points are obtained, each of which has a feature value greater than the first effective threshold, where Y is a positive integer. If the gradient value of the zth candidate feature point among the Y candidate feature points is greater than or equal to the gradient value of each first original feature point in the set of adjacent feature points of the zth candidate feature point, then the zth candidate feature point is determined as the first key feature point of the template image; accordingly, if there is a first original feature point in the set of adjacent feature points of the zth candidate feature point whose gradient value is greater than the gradient value of the zth candidate feature point, then the zth candidate feature point is discarded, where z is a positive integer and is less than or equal to Y. Furthermore, if the number of the first key feature points is greater than the preset number of feature points W (the preset number of feature points W is determined according to the complexity of the image in the template image, and the number of feature points W is proportional to the complexity), the step range of the z-th candidate feature point is expanded (that is, the range of the adjacent area of ​​the z-th candidate feature point is expanded, such as expanding the adjacent area from 3 to 3 expanded to 5 5), until the number of the first key feature points is close to the preset number of feature points W. Similarly, if the number of the first key feature points is less than the preset number of feature points W, then the step range of the zth candidate feature point is reduced (i.e., the range of the adjacent area of ​​the zth candidate feature point is reduced, such as the adjacent area from 5 5 expanded to 3 3) until the number of first key feature points approaches the preset number of feature points W. Optionally, the Y candidate feature points are sampled by using spacing sampling, sorting sampling, random sampling, etc. to obtain M first key feature points.

[0073] S404: Adjust the original gradient information of the M first key feature points to obtain the gradient information of the M first key feature points.

[0074] In one implementation, the gradient values ​​of the M first key feature points are normalized (i.e., the gradient values ​​of the M first key feature points are set to unit values). Furthermore, a first quantization standard direction set is obtained, the first quantization standard direction set including at least two first quantization standard directions. The at least two first quantization standard directions included in the first quantization standard direction set can be pre-set or specified by the user (e.g., specifying four directions of up, down, left, and right as the first quantization standard directions); using the first quantization standard direction in the first quantization standard direction set as a reference direction, the original gradient directions of the M first key feature points are quantized to obtain the quantization reference directions of the M first key feature points; the quantization reference directions of the M first key feature points are adjusted to determine the gradient directions of each first key feature point; wherein the quantization direction of the i-th first key feature point among the M first key feature points is determined based on the quantization reference direction of the i-th first key feature point and the quantization reference directions of each first key feature point in the set of adjacent first key feature points of the i-th first key feature point, where i is a positive integer.

[0075] Taking the jth first key feature point among M first key feature points as an example, the first quantization standard direction in the first quantization standard direction set is used as a reference direction, and the specific implementation method for quantizing the original gradient directions of the M first key feature points is as follows: calculating the angle information between the original gradient direction of the jth first key feature point and one or more first quantization standard directions in the first quantization standard direction set, and determining the target first quantization standard direction as the quantization reference direction of the jth first key feature point, where j is a positive integer and j is less than or equal to M; wherein, among the calculated angle information, the angle value indicated by the angle information between the target first quantization standard direction and the original gradient direction of the jth first key feature point is the smallest (i.e., the proximity principle). In one embodiment, the absolute value of the difference between the original gradient direction of the jth first key feature point and one or more first quantization standard directions in the first quantization standard direction set is calculated to obtain the angle information.

[0076] For example, assume that the first quantization standard direction set includes eight first quantization standard directions d1-d8, specifically: d1 = 0°, d2 = 45°, d3 = 90°, d4 = 135°, d5 = 180°, d6 = 225°, d7 = 270°, and d8 = 315°. The gradient direction of the first key feature point a is 10°, the gradient direction of the first key feature point b is 110°, and the gradient direction of the first key feature point c is 40°. Since the gradient direction of the first key feature point a has the smallest angle with the first quantization standard direction d1, the quantization reference direction of the first key feature point a is set to 0°. Similarly, the gradient direction of the first key feature point b has the smallest angle with the first quantization standard direction d3, so the quantization reference direction of the first key feature point b is set to 90°. The gradient direction of the first key feature point c has the smallest angle with the first quantization standard direction d2, so the quantization reference direction of the first key feature point c is set to 45°.

[0077] In one embodiment, the specific implementation method of adjusting the quantization reference directions of the M first key feature points and determining the gradient direction of each first key feature point is as follows: the quantization direction that appears most frequently in the kth first key feature point among the M first key feature points and in each first key feature point in a set of neighboring feature points of the kth first key feature point is determined as the gradient direction of the kth first key feature point, where k is a positive integer and k is less than or equal to M. The neighboring feature points of the kth first key feature point may be feature points that are separated from the kth first key feature point by a step length of L, where L is a positive integer. Figure 5c A schematic diagram of adjacent feature points provided in an embodiment of the present application is shown in FIG. Figure 5c As shown, when L=1, the adjacent feature points of the kth first key feature point are feature point f1-feature point f8; optionally, in some embodiments, the adjacent feature points of the kth first key feature point may also be feature point f2, feature point f4, feature point f6, and feature point f8. Assume that the quantization reference direction of the kth first key feature point is 45°, and among feature points f1-feature point f8, feature point f2 and feature point f5 are first key feature points, and the quantization reference direction of feature point f2 is 45°, and the quantization reference direction of feature point f5 is 60°, then 45° is determined as the gradient direction of the kth first key feature point.

[0078] Optionally, the quantization direction of the vth first key feature point among the M first key feature points and the quantization direction of each first key feature point in the set of adjacent feature points of the vth first key feature point are added to the gradient direction set of the vth first key feature point to obtain the gradient direction set of the vth first key feature point, where v is a positive integer.

[0079] S405: Perform feature extraction on the image to be matched to obtain gradient information of Q second original feature points of the image to be matched.

[0080] In one embodiment, feature extraction from the image to be matched includes performing edge detection processing on the image to be matched using an edge detection algorithm. Specifically, a gradient map corresponding to the image to be matched is calculated using the edge detection algorithm. The gradient map includes Q second original feature points, and gradient information (original gradient values ​​and original gradient directions) of the Q second original feature points is obtained using the gradient map. The edge detection algorithm may include, but is not limited to, a local difference (Roberts) operator, a pixel average (Prewitt) operator, and a weighted average (Sobel) operator.

[0081] S406: Determine N second key feature points from the Q second original feature points according to the second effective threshold.

[0082] In one embodiment, Q second original feature points are filtered using a second effective threshold to obtain N second key feature points, each of which has a feature value greater than the second effective threshold. For example, assuming the second effective threshold is 3, the feature value of original feature point 1 is 5, the feature value of original feature point 2 is 2, and the feature value of original feature point 3 is 8; then, original feature points 1-3 are filtered using the second effective threshold to obtain two second key feature points (i.e., original feature point 1 and original feature point 3). Filtering the second original feature points using the second effective threshold can filter out noise feature points in the second original feature points, thereby enhancing the robustness of the image matching results.

[0083] Optionally, sampling processing is performed on the image to be matched. Specifically, if the gradient value of the sth second original feature point among the Q second original feature points is greater than the second effective threshold, and the gradient value of the sth second original feature point is greater than or equal to the gradient value of each second original feature point in the set of adjacent second original feature points of the sth original feature point, then the sth original feature point is determined as the second key feature point of the image to be matched, where s is a positive integer and s is less than or equal to Q.

[0084] S407: Determine a gradient direction set of N second key feature points according to the original gradient directions of the N second key feature points.

[0085] Obtain a second quantization standard direction set, the second quantization standard direction set including at least two second quantization standard directions; use the second quantization standard direction in the second quantization standard direction set as a reference direction, quantize the original gradient directions of N second key feature points, and obtain the quantization reference directions of the N second key feature points; determine the gradient direction set of the N second key feature points based on the quantization reference directions of the N second key feature points; wherein the gradient direction set of the x-th second key feature point includes the quantization reference direction of the x-th second key feature point and the quantization reference directions of each second key feature point in a set of adjacent feature points of the x-th second key feature point, where x is a positive integer and x is less than or equal to N.

[0086] Taking the yth second key feature point among the N second key feature points as an example, the second quantization standard direction in the second quantization standard direction set is used as a reference direction, and the specific implementation method of quantizing the original gradient directions of the N first key feature points is: calculating the angle information between the original gradient direction of the yth second key feature point and one or more second quantization standard directions in the second quantization standard direction set; determining the target second quantization standard direction as the quantization reference direction of the yth second key feature point, y is a positive integer, and y is less than or equal to N; wherein, in the calculated angle information, the angle value indicated by the angle information between the target second quantization standard direction and the original gradient direction of the yth second key feature point is the smallest (that is, the proximity principle).

[0087] In one embodiment, a specific implementation method for determining the gradient direction set of N second key feature points based on the quantization reference directions of the N second key feature points is: adding the quantization direction of the p-th second key feature point among the N second key feature points and the quantization direction of each second key feature point in the adjacent feature point set of the p-th second key feature point to the gradient direction set of the p-th second key feature point to obtain the gradient direction set of the p-th second key feature point, where p is a positive integer. Figure 5d This is a schematic diagram of determining a gradient direction set provided in an embodiment of the present application. Figure 5d As shown, with feature point A as the center 3 The range of 3 includes direction 1 and direction 2, so the gradient direction set of feature point A includes direction 1 and direction 2; similarly, with feature point B as the center, 3 The range of 3 includes direction 1, direction 2, direction 3 and direction 4, so the gradient direction set of feature point B includes direction 1-direction 4.

[0088] S408: Calculate similarity information of the P sub-regions in the to-be-matched image based on the gradient information of the M first key feature points and the gradient information of the second key feature points included in each sub-region of the to-be-matched image.

[0089] S409: Determine a target sub-region from the P sub-regions according to the similarity information of the P sub-regions.

[0090] The specific implementation of step S408 and step S409 can be referred to Figure 2 The implementation of step S204 and step S205 will not be repeated here. Figure 5e A matching result diagram provided in an embodiment of the present application is shown as follows: Figure 5e As shown, the matching result is shown in Figure 1. Figure 5a The template diagram shown and Figure 5b The mask image shown is the result obtained after image matching processing is performed on the image to be matched.

[0091] Figure 5f This is an architecture diagram of an image matching provided in an embodiment of the present application. Figure 5f As shown, the dotted line part is the path for the smart device to process the template image: first, the smart device extracts features from the template image (that is, calculates the gradient value and gradient direction of the template image, and filters according to the gradient value), then samples feature points of the template image after feature extraction (that is, samples using the domain maximum method), and then adjusts the gradient information of the sampled feature points (normalizes the gradient value and quantizes the gradient direction), and uses the mask image to filter the adjusted template image to obtain the template feature map; the solid line part is the path for the smart device to process the matching image: first, the smart device extracts features from the matching image (that is, calculates the gradient value and gradient direction of the image to be matched, and filtering by gradient value), then adjusting the gradient information of the feature points of the image to be matched after feature extraction (normalizing the gradient value and quantizing the gradient direction), and obtaining the gradient direction set of each feature point through diffusion operation (i.e., adding the gradient direction of the adjacent feature points of the target feature point to the gradient direction set of the target feature point), and then obtaining the feature map to be matched; performing similarity measurement on the template feature map and the feature map to be matched (i.e., calculating the similarity between each sub-region in the image to be matched and the template map); optionally, limiting the matching area in the image to be matched, and using a sliding window to determine the matching result in the limited matching area (e.g., searching for the target sub-region with the highest similarity to the template map).

[0092] The embodiments of this application are Figure 2 On the basis of the embodiment, by sampling feature points of the template image, the number of feature points is reduced, thereby reducing the matching complexity and improving the matching efficiency. The sampled feature points are more stable, which helps to improve the stability of image matching. In addition, the strong constraints on the feature point positions are relaxed through the diffusion operation, which can effectively suppress texture edge interference, thereby improving the robustness of the algorithm.

[0093] The above describes in detail the method of the embodiment of the present application. In order to facilitate better implementation of the above scheme of the embodiment of the present application, the device of the embodiment of the present application is provided below accordingly.

[0094] See Figure 6 , Figure 6 This is a schematic diagram of the structure of an image matching device provided in an embodiment of the present application. The device can be mounted on Figure 1 On the smart device 102 shown. Figure 6 The image matching device shown can be used to perform the above Figure 2 and Figure 4 Part or all of the functions of the method embodiment described. The detailed description of each unit is as follows:

[0095] An acquisition unit 601 is configured to acquire a to-be-matched image and a template image corresponding to the to-be-matched image;

[0096] A processing unit 602 is configured to perform feature extraction on the template image to obtain gradient information of M first key feature points of the template image, where the gradient information of the first key feature points includes a gradient value and a gradient direction of the first key feature points, and M is a positive integer.

[0097] and extracting features from the image to be matched to obtain gradient information of N second key feature points of the image to be matched, wherein the gradient information of the second key feature points includes a gradient value and a gradient direction set of the second key feature points, where N is a positive integer;

[0098] and for calculating similarity information of P sub-regions in the to-be-matched image based on the gradient information of the M first key feature points and the gradient information of the second key feature points included in each sub-region of the to-be-matched image, where P is a positive integer;

[0099] and for determining a target subregion from the P subregions according to the similarity information of the P subregions;

[0100] Among them, the similarity of each sub-region in the to-be-matched image is obtained based on the target feature points in each sub-region in the to-be-matched image; the target feature points belong to the N second key feature points, and the gradient direction set of the target feature points contains the gradient direction of the first key feature point corresponding to the target feature point.

[0101] In one embodiment, the processing unit 602 is specifically configured to:

[0102] Performing feature extraction on the template image to obtain gradient information of K first original feature points of the template image, where K is a positive integer;

[0103] Determining M first key feature points from the K first original feature points according to a first effective threshold, wherein feature values ​​of the M first key feature points are greater than the first effective threshold;

[0104] The gradient values ​​of the M first key feature points are set to unit values, and the gradient directions of the M first key feature points are calibrated to obtain gradient information of the M first key feature points of the template image.

[0105] In one embodiment, the processing unit 602 is specifically configured to:

[0106] Acquire a first quantization standard direction set by an acquisition unit 601, where the first quantization standard direction set includes at least two first quantization standard directions;

[0107] Taking a first quantization standard direction in the first quantization standard direction set as a reference direction, quantizing the original gradient directions of the M first key feature points to obtain quantization reference directions of the M first key feature points;

[0108] Adjusting the quantization reference directions of the M first key feature points to determine the gradient direction of each first key feature point;

[0109] Among them, the quantization direction of the i-th first key feature point among the M first key feature points is determined based on the quantization reference direction of the i-th first key feature point and the quantization reference directions of each first key feature point in the set of adjacent first key feature points of the i-th first key feature point, and i is a positive integer.

[0110] In one embodiment, the processing unit 602 is specifically configured to:

[0111] Calculating angle information between an original gradient direction of a j-th first key feature point among the M first key feature points and one or more first quantization standard directions in the first quantization standard direction set;

[0112] Determine the target first quantization standard direction as the quantization reference direction of the j-th first key feature point, where j is a positive integer and is less than or equal to M;

[0113] Among the calculated angle information, the angle value indicated by the angle information between the target first quantization standard direction and the original gradient direction of the j-th first key feature point is the smallest.

[0114] In one embodiment, the processing unit 602 is specifically configured to:

[0115] Acquiring a mask image that matches the template image through an acquisition unit 601, wherein the mask image includes a foreground area;

[0116] Determining an area in the template image that matches the foreground area as a valid area of ​​the template image;

[0117] Feature extraction is performed on the effective area of ​​the template image to obtain gradient information of K first original feature points of the template image.

[0118] In one embodiment, the processing unit 602 is specifically configured to:

[0119] If, among the K first original feature points, the gradient value of the rth first original feature point is greater than the first valid threshold, and the gradient value of the rth first original feature point is greater than or equal to the gradient value of each first original feature point in the set of adjacent feature points of the rth original feature point, then the rth original feature point is determined as the first key feature point of the template image, where r is a positive integer and is less than or equal to K.

[0120] In one embodiment, the processing unit 602 is specifically configured to:

[0121] Perform feature extraction on the image to be matched to obtain gradient information of Q second original feature points of the image to be matched, where Q is a positive integer;

[0122] Determining N second key feature points from the Q second original feature points according to a second effective threshold, wherein feature values ​​of the N second key feature points are greater than the second effective threshold;

[0123] A gradient direction set of the N second key feature points is determined according to the original gradient directions of the N second key feature points.

[0124] In one embodiment, the processing unit 602 is specifically configured to:

[0125] Acquire a second quantization standard direction set by an acquisition unit 601, where the second quantization standard direction set includes at least two second quantization standard directions;

[0126] Using a second quantization standard direction in the second quantization standard direction set as a reference direction, quantizing the original gradient directions of the N second key feature points to obtain quantization reference directions of the N second key feature points;

[0127] Determining a gradient direction set of the N second key feature points according to the quantization reference directions of the N second key feature points;

[0128] The gradient direction set of the xth second key feature point includes the quantized reference direction of the xth second key feature point and the quantized reference direction of each second key feature point in the adjacent feature point set of the xth second key feature point, where x is a positive integer.

[0129] In one embodiment, the processing unit 602 is specifically configured to:

[0130] Calculating angle information between an original gradient direction of the yth second key feature point among the N second key feature points and one or more second quantization standard directions in the second quantization standard direction set;

[0131] Determine the target second quantization standard direction as the quantization reference direction of the y-th second key feature point, where y is a positive integer and is less than or equal to N;

[0132] Among the angle information obtained by calculation, the angle value indicated by the angle information between the target second quantization standard direction and the original gradient direction of the y-th second key feature point is the smallest.

[0133] In one embodiment, the processing unit 602 is specifically configured to:

[0134] If, among the Q second original feature points, the gradient value of the sth second original feature point is greater than the second valid threshold, and the gradient value of the sth second original feature point is greater than or equal to the gradient value of each second original feature point in the set of adjacent second original feature points of the sth original feature point, then the sth original feature point is determined as the second key feature point of the image to be matched, s is a positive integer, and s is less than or equal to Q.

[0135] In one embodiment, each sub-region in the to-be-matched image is obtained by sliding a sliding window through the to-be-matched image; the size of the sliding window is the same as that of the template image.

[0136] According to one embodiment of the present application, Figure 2 and Figure 4 Some of the steps involved in the image matching method shown can be represented by Figure 6 The image matching device shown is executed by each unit. For example, Figure 2 The step S201 shown in FIG. Figure 6 The acquisition unit 601 shown in FIG. 6 is executed, and steps S202 to S205 can be performed by Figure 6 Processing unit 602 is shown executing. Figure 4 The step S401 shown in FIG. Figure 6 The acquisition unit 601 shown in FIG. 6 is executed, and steps S402 to S409 can be performed by Figure 6 Processing unit 602 is shown executing. Figure 6 The various units in the image matching device shown can be individually or all combined into one or several other units to form a whole, or one (or some) of the units can be further divided into multiple functionally smaller units to form a whole, which can achieve the same operation without affecting the realization of the technical effects of the embodiments of the present application. The above-mentioned units are divided based on logical functions. In actual applications, the functions of one unit can also be implemented by multiple units, or the functions of multiple units can be implemented by one unit. In other embodiments of the present application, the image matching device may also include other units. In actual applications, these functions can also be implemented with the assistance of other units, and can be implemented by the collaboration of multiple units.

[0137] According to another embodiment of the present application, the program can be executed by running on a general computing device such as a computer including a central processing unit (CPU), a random access memory (RAM), a read-only memory (ROM) and other processing elements and storage elements. Figure 2 and Figure 4 The computer program (including program code) of each step involved in the corresponding method shown in is constructed as follows Figure 6 The image matching device shown in and the image matching method of the embodiment of the present application are implemented. The computer program can be recorded on a computer-readable recording medium, for example, and loaded into the above-mentioned computing device through the computer-readable recording medium and run therein.

[0138] Based on the same inventive concept, the principles and beneficial effects of the image matching device provided in the embodiment of the present application for solving the problem are similar to the principles and beneficial effects of the image matching device in the method embodiment of the present application. Please refer to the principles and beneficial effects of the implementation of the method. For the sake of concise description, they will not be repeated here.

[0139] See also Figure 7 , Figure 7This is a schematic diagram of the structure of a smart device provided in an embodiment of the present application. The smart device includes at least a processor 701, a communication interface 702, and a memory 703. The processor 701, communication interface 702, and memory 703 may be connected via a bus or other means. The processor 701 (also known as the Central Processing Unit (CPU)) is the computing and control core of the terminal. It can interpret various commands within the terminal and process various data within the terminal. For example, the CPU can interpret power on / off commands sent by the user to the terminal and control the terminal's power on / off operation. Another example is that the CPU can transmit various interactive data between the terminal's internal components. The communication interface 702 may optionally include a standard wired interface or a wireless interface (such as Wi-Fi or a mobile communication interface). Under the control of the processor 701, it can be used to send and receive data. The communication interface 702 can also be used for data transmission and interaction within the terminal. The memory 703 is a storage device within the terminal, used to store programs and data. It is understood that the memory 703 herein can include both the terminal's built-in memory and the terminal's supported extended memory. The memory 703 provides a storage space that stores the terminal's operating system, which may include but is not limited to: Android system, iOS system, Windows Phone system, etc., and this application does not limit this.

[0140] In the embodiment of the present application, the processor 701 is configured to perform the following operations by running the executable program code in the memory 703:

[0141] Obtaining a to-be-matched graph and a template graph corresponding to the to-be-matched graph through the communication interface 702;

[0142] Performing feature extraction on the template image to obtain gradient information of M first key feature points of the template image, where the gradient information of the first key feature points includes a gradient value and a gradient direction of the first key feature points, where M is a positive integer;

[0143] Performing feature extraction on the image to be matched to obtain gradient information of N second key feature points of the image to be matched, where the gradient information of the second key feature points includes a gradient value and a gradient direction set of the second key feature points, where N is a positive integer;

[0144] Calculate similarity information of P sub-regions in the to-be-matched image based on the gradient information of the M first key feature points and the gradient information of the second key feature points included in each sub-region of the to-be-matched image, where P is a positive integer;

[0145] Determining a target subregion from the P subregions according to the similarity information of the P subregions;

[0146] Among them, the similarity of each sub-region in the to-be-matched image is obtained based on the target feature points in each sub-region in the to-be-matched image; the target feature points belong to the N second key feature points, and the gradient direction set of the target feature points contains the gradient direction of the first key feature point corresponding to the target feature point.

[0147] As an optional embodiment, the processor 701 performs feature extraction on the template image to obtain gradient information of M first key feature points of the template image. A specific embodiment is as follows:

[0148] Performing feature extraction on the template image to obtain gradient information of K first original feature points of the template image, where K is a positive integer;

[0149] Determining M first key feature points from the K first original feature points according to a first effective threshold, wherein feature values ​​of the M first key feature points are greater than the first effective threshold;

[0150] The gradient values ​​of the M first key feature points are set to unit values, and the gradient directions of the M first key feature points are calibrated to obtain gradient information of the M first key feature points of the template image.

[0151] As an optional embodiment, the processor 701 performs calibration processing on the gradient directions of the M first key feature points to obtain the gradient information of the M first key feature points of the template image. A specific embodiment is as follows:

[0152] Acquire a first quantization standard direction set through the communication interface 702, where the first quantization standard direction set includes at least two first quantization standard directions;

[0153] Taking a first quantization standard direction in the first quantization standard direction set as a reference direction, quantizing the original gradient directions of the M first key feature points to obtain quantization reference directions of the M first key feature points;

[0154] Adjusting the quantization reference directions of the M first key feature points to determine the gradient direction of each first key feature point;

[0155] Among them, the quantization direction of the i-th first key feature point among the M first key feature points is determined based on the quantization reference direction of the i-th first key feature point and the quantization reference directions of each first key feature point in the set of adjacent first key feature points of the i-th first key feature point, and i is a positive integer.

[0156] As an optional embodiment, the processor 701 uses the first quantization standard direction in the first quantization standard direction set as a reference direction to quantize the original gradient directions of the M first key feature points. A specific embodiment of obtaining the quantization reference directions of the M first key feature points is as follows:

[0157] Calculating angle information between an original gradient direction of a j-th first key feature point among the M first key feature points and one or more first quantization standard directions in the first quantization standard direction set;

[0158] Determine the target first quantization standard direction as the quantization reference direction of the j-th first key feature point, where j is a positive integer and is less than or equal to M;

[0159] Among the calculated angle information, the angle value indicated by the angle information between the target first quantization standard direction and the original gradient direction of the j-th first key feature point is the smallest.

[0160] As an optional embodiment, the processor 701 performs feature extraction on the template image to obtain gradient information of K feature points of the template image. A specific embodiment is as follows:

[0161] Acquire a mask image matching the template image through the communication interface 702, wherein the mask image includes a foreground area;

[0162] Determining an area in the template image that matches the foreground area as a valid area of ​​the template image;

[0163] Feature extraction is performed on the effective area of ​​the template image to obtain gradient information of K first original feature points of the template image.

[0164] As an optional embodiment, a specific embodiment in which the processor 701 determines M first key feature points from the K first original feature points according to the first effective threshold is:

[0165] If, among the K first original feature points, the gradient value of the rth first original feature point is greater than the first valid threshold, and the gradient value of the rth first original feature point is greater than or equal to the gradient value of each first original feature point in the set of adjacent feature points of the rth original feature point, then the rth original feature point is determined as the first key feature point of the template image, where r is a positive integer and is less than or equal to K.

[0166] As an optional embodiment, the processor 701 performs feature extraction on the image to be matched, and obtains the gradient information of N second key feature points of the image to be matched. A specific embodiment is as follows:

[0167] Perform feature extraction on the image to be matched to obtain gradient information of Q second original feature points of the image to be matched, where Q is a positive integer;

[0168] Determining N second key feature points from the Q second original feature points according to a second effective threshold, wherein feature values ​​of the N second key feature points are greater than the second effective threshold;

[0169] A gradient direction set of the N second key feature points is determined according to the original gradient directions of the N second key feature points.

[0170] As an optional embodiment, a specific embodiment in which the processor 701 determines the gradient direction set of the N second key feature points of the to-be-matched image according to the original gradient directions of the N second key feature points is:

[0171] Acquire a second quantization standard direction set through the communication interface 702, where the second quantization standard direction set includes at least two second quantization standard directions;

[0172] Using a second quantization standard direction in the second quantization standard direction set as a reference direction, quantizing the original gradient directions of the N second key feature points to obtain quantization reference directions of the N second key feature points;

[0173] Determining a gradient direction set of the N second key feature points according to the quantization reference directions of the N second key feature points;

[0174] The gradient direction set of the xth second key feature point includes the quantized reference direction of the xth second key feature point and the quantized reference direction of each second key feature point in the adjacent feature point set of the xth second key feature point, where x is a positive integer.

[0175] As an optional embodiment, the processor 701 uses the second quantization standard direction in the second quantization standard direction set as a reference direction to quantize the original gradient directions of the N second key feature points. A specific embodiment of obtaining the quantization reference directions of the N second key feature points is as follows:

[0176] Calculating angle information between an original gradient direction of the yth second key feature point among the N second key feature points and one or more second quantization standard directions in the second quantization standard direction set;

[0177] Determine the target second quantization standard direction as the quantization reference direction of the y-th second key feature point, where y is a positive integer and is less than or equal to N;

[0178] Among the angle information obtained by calculation, the angle value indicated by the angle information between the target second quantization standard direction and the original gradient direction of the y-th second key feature point is the smallest.

[0179] As an optional embodiment, a specific embodiment in which the processor 701 determines N second key feature points from the Q second original feature points according to the second effective threshold is:

[0180] If, among the Q second original feature points, the gradient value of the sth second original feature point is greater than the second valid threshold, and the gradient value of the sth second original feature point is greater than or equal to the gradient value of each second original feature point in the set of adjacent second original feature points of the sth original feature point, then the sth original feature point is determined as the second key feature point of the image to be matched, s is a positive integer, and s is less than or equal to Q.

[0181] As an optional embodiment, each sub-region in the to-be-matched image is obtained by slidingly traversing the to-be-matched image using a sliding window; the size of the sliding window is the same as that of the template image.

[0182] Based on the same inventive concept, the principles and beneficial effects of solving the problems provided by the smart device in the embodiment of the present application are similar to the principles and beneficial effects of solving the problems by the image matching method in the method embodiment of the present application. Please refer to the principles and beneficial effects of the implementation of the method. For the sake of concise description, they will not be repeated here.

[0183] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores one or more instructions, and the one or more instructions are suitable for being loaded by a processor and executing the image matching method described in the above method embodiment.

[0184] An embodiment of the present application further provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute the image matching method described in the above method embodiment.

[0185] The present application also provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the above-described image matching method.

[0186] The steps in the method of the embodiment of the present application can be adjusted in order, combined and deleted according to actual needs.

[0187] The modules in the device of the embodiment of the present application can be merged, divided and deleted according to actual needs.

[0188] Those skilled in the art will appreciate that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium, which can include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc.

[0189] The above disclosure is only a preferred embodiment of the present application, and certainly cannot be used to limit the scope of rights of the present application. Ordinary technicians in this field can understand that implementing all or part of the processes of the above embodiment and making equivalent changes in accordance with the claims of this application still fall within the scope of the invention.

Claims

1. An image matching method, characterized in that: The method comprises: Acquire the collected image to be matched and the template image corresponding to the image to be matched; the template image is used to indicate the object to be searched in the image to be matched; Performing feature extraction on the template image to obtain gradient information of M first key feature points of the template image, where the gradient information of the first key feature points includes a gradient value and a gradient direction of the first key feature point, and M is a positive integer; wherein the gradient information of the M first key feature points is used to indicate features of a predicted edge region in the template image, and the performing feature extraction on the template image includes performing edge detection processing on the template image using an edge detection algorithm, and screening and calibrating a result obtained by the edge detection processing to obtain the gradient information of the M first key feature points in the template image; Performing feature extraction on the image to be matched to obtain gradient information of N second key feature points of the image to be matched, where the gradient information of the second key feature points includes a gradient value and a gradient direction set of the second key feature points, where N is a positive integer; wherein the gradient information of the N second key feature points is used to indicate edge features of one or more objects in the image to be matched, and the performing feature extraction on the image to be matched includes performing edge detection processing on an object in the image to be matched using an edge detection algorithm to obtain gradient information of the N second key feature points of the object; Calculate similarity information of P sub-regions in the to-be-matched image based on the gradient information of the M first key feature points and the gradient information of the second key feature points included in each sub-region of the to-be-matched image, where P is a positive integer; Determining a target subregion from the P subregions according to the similarity information of the P subregions; Among them, the similarity of each sub-region in the to-be-matched image is obtained based on the target feature points in each sub-region in the to-be-matched image; the target feature points belong to the N second key feature points, and the gradient direction set of the target feature points contains the gradient direction of the first key feature point corresponding to the target feature point. The similarity information of the P sub-regions is obtained by counting the number of target feature points contained in each sub-region, and the similarity of each sub-region is proportional to the number of target feature points.

2. The method according to claim 1, wherein The step of extracting features from the template image to obtain gradient information of M first key feature points of the template image includes: Performing feature extraction on the template image to obtain gradient information of K first original feature points of the template image, where K is a positive integer; Determining M first key feature points from the K first original feature points according to a first effective threshold, wherein feature values ​​of the M first key feature points are greater than the first effective threshold; The gradient values ​​of the M first key feature points are set to unit values, and the gradient directions of the M first key feature points are calibrated to obtain gradient information of the M first key feature points of the template image.

3. The method according to claim 2, wherein The calibrating the gradient directions of the M first key feature points to obtain gradient information of the M first key feature points of the template image includes: Acquire a first quantization standard direction set, where the first quantization standard direction set includes at least two first quantization standard directions; Taking a first quantization standard direction in the first quantization standard direction set as a reference direction, quantizing the original gradient directions of the M first key feature points to obtain quantization reference directions of the M first key feature points; Adjusting the quantization reference directions of the M first key feature points to determine the gradient direction of each first key feature point; Among them, the quantization direction of the i-th first key feature point among the M first key feature points is determined based on the quantization reference direction of the i-th first key feature point and the quantization reference directions of each first key feature point in the set of adjacent first key feature points of the i-th first key feature point, and i is a positive integer.

4. The method according to claim 3, wherein The step of taking the first quantization standard direction in the first quantization standard direction set as a reference direction and performing quantization processing on the original gradient directions of the M first key feature points to obtain the quantization reference directions of the M first key feature points includes: Calculating angle information between an original gradient direction of a j-th first key feature point among the M first key feature points and one or more first quantization standard directions in the first quantization standard direction set; Determine the target first quantization standard direction as the quantization reference direction of the j-th first key feature point, where j is a positive integer and is less than or equal to M; Among the calculated angle information, the angle value indicated by the angle information between the target first quantization standard direction and the original gradient direction of the j-th first key feature point is the smallest.

5. The method according to claim 2, wherein The step of extracting features from the template image to obtain gradient information of K feature points of the template image includes: Acquire a mask image that matches the template image, wherein the mask image includes a foreground area; Determining an area in the template image that matches the foreground area as a valid area of ​​the template image; Feature extraction is performed on the effective area of ​​the template image to obtain gradient information of K first original feature points of the template image.

6. The method according to claim 2, wherein The step of determining M first key feature points from the K first original feature points according to the first effective threshold comprises: If, among the K first original feature points, the gradient value of the rth first original feature point is greater than the first valid threshold, and the gradient value of the rth first original feature point is greater than or equal to the gradient value of each first original feature point in the set of adjacent feature points of the rth original feature point, then the rth original feature point is determined as the first key feature point of the template image, where r is a positive integer and is less than or equal to K.

7. The method according to claim 1, wherein The step of extracting features from the image to be matched to obtain gradient information of N second key feature points of the image to be matched includes: Perform feature extraction on the image to be matched to obtain gradient information of Q second original feature points of the image to be matched, where Q is a positive integer; Determining N second key feature points from the Q second original feature points according to a second effective threshold, wherein feature values ​​of the N second key feature points are greater than the second effective threshold; A gradient direction set of the N second key feature points is determined according to the original gradient directions of the N second key feature points.

8. The method according to claim 7, wherein The step of determining a set of gradient directions of the N second key feature points of the to-be-matched image based on the original gradient directions of the N second key feature points includes: Acquire a second quantization standard direction set, where the second quantization standard direction set includes at least two second quantization standard directions; Using a second quantization standard direction in the second quantization standard direction set as a reference direction, quantizing the original gradient directions of the N second key feature points to obtain quantization reference directions of the N second key feature points; Determining a gradient direction set of the N second key feature points according to the quantization reference directions of the N second key feature points; The gradient direction set of the xth second key feature point includes the quantized reference direction of the xth second key feature point and the quantized reference direction of each second key feature point in the adjacent feature point set of the xth second key feature point, where x is a positive integer.

9. The method according to claim 8, wherein The step of quantizing the original gradient directions of the N second key feature points using the second quantization standard direction in the second quantization standard direction set as a reference direction to obtain the quantization reference directions of the N second key feature points includes: Calculating angle information between an original gradient direction of the yth second key feature point among the N second key feature points and one or more second quantization standard directions in the second quantization standard direction set; Determine the target second quantization standard direction as the quantization reference direction of the y-th second key feature point, where y is a positive integer and is less than or equal to N; Among the angle information obtained by calculation, the angle value indicated by the angle information between the target second quantization standard direction and the original gradient direction of the y-th second key feature point is the smallest.

10. The method according to claim 7, wherein: The step of determining N second key feature points from the Q second original feature points according to the second effective threshold comprises: If, among the Q second original feature points, the gradient value of the sth second original feature point is greater than the second valid threshold, and the gradient value of the sth second original feature point is greater than or equal to the gradient value of each second original feature point in the set of adjacent second original feature points of the sth original feature point, then the sth original feature point is determined as the second key feature point of the image to be matched, s is a positive integer, and s is less than or equal to Q.

11. The method according to claim 1, wherein Each sub-region in the to-be-matched image is obtained by sliding a sliding window through the to-be-matched image; the size of the sliding window is the same as that of the template image.

12. An image matching device, characterized in that: The image matching device comprises: An acquisition unit, configured to acquire the collected image to be matched and a template image corresponding to the image to be matched; the template image is used to indicate an object to be searched in the image to be matched; a processing unit, configured to perform feature extraction on the template image to obtain gradient information of M first key feature points of the template image, where the gradient information of the first key feature points includes a gradient value and a gradient direction of the first key feature point, and M is a positive integer; wherein the gradient information of the M first key feature points is used to indicate features of a predicted edge region in the template image, and the feature extraction on the template image includes performing edge detection processing on the template image using an edge detection algorithm, and screening and calibrating a result obtained by the edge detection processing to obtain the gradient information of the M first key feature points in the template image; and for performing feature extraction on the image to be matched to obtain gradient information of N second key feature points of the image to be matched, where the gradient information of the second key feature points includes a gradient value and a gradient direction set of the second key feature points, where N is a positive integer; wherein the gradient information of the N second key feature points is used to indicate edge features of one or more objects in the image to be matched, and the feature extraction on the image to be matched includes performing edge detection processing on the objects in the image to be matched using an edge detection algorithm to obtain gradient information of the N second key feature points of the objects; and for calculating similarity information of P sub-regions in the to-be-matched image based on the gradient information of the M first key feature points and the gradient information of the second key feature points included in each sub-region of the to-be-matched image, where P is a positive integer; and for determining a target subregion from the P subregions according to the similarity information of the P subregions; Among them, the similarity of each sub-region in the to-be-matched image is obtained based on the target feature points in each sub-region in the to-be-matched image; the target feature points belong to the N second key feature points, and the gradient direction set of the target feature points contains the gradient direction of the first key feature point corresponding to the target feature point. The similarity information of the P sub-regions is obtained by counting the number of target feature points contained in each sub-region, and the similarity of each sub-region is proportional to the number of target feature points.

13. A smart device, characterized in that: include: storage devices and processors; The storage device stores a computer program; A processor executes a computer program to implement the image matching method according to any one of claims 1 to 11.

14. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program. When the computer program is executed by a processor, the image matching method according to any one of claims 1 to 11 is implemented.

15. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instruction is executed by a processor, the image matching method according to any one of claims 1 to 11 is implemented.

Images