Image matching method, device, equipment and medium

Abstract

The present disclosure provides an image matching method, device, equipment and medium, wherein the image matching method comprises: obtaining a to-be-matched image; extracting corresponding main feature points of the to-be-matched image according to a gradient direction angle, and determining a descriptor of the main feature points; obtaining a target template image, and performing image matching on the to-be-matched image and the target template image based on the descriptor of the main feature points. By using the above scheme, the main feature points are extracted according to the gradient direction of the to-be-matched image, and the descriptor of the main feature points is determined, the image matching is performed on the to-be-matched image and the target template image based on the descriptor of the main feature points, the main feature points have rich feature information, the matching accuracy can be ensured when the to-be-matched image is a weak texture image, the complexity and resource consumption of image matching are reduced, the matching efficiency is improved, and two images can be quickly matched.

Description

Technical Field

[0001] This disclosure relates to the field of image processing technology, and in particular to an image matching method, apparatus, device and medium. Background Technology

[0002] With the continuous development of computer technology, more and more interactive scenarios are being combined with computer technology, greatly enhancing the fun and technological feel of the interaction.

[0003] In many interactive scenarios, such as paper-screen interaction, it is necessary to align or match two images. Related technologies can use feature matching algorithms, template matching algorithms, or deep network algorithms for image matching. However, when the image is a weakly textured image with few feature points, feature matching algorithms have low matching accuracy, while template matching algorithms and deep network algorithms have high complexity and resource consumption, resulting in low matching efficiency. In summary, related image matching technologies cannot simultaneously meet the requirements in terms of matching accuracy, complexity, resource consumption, and matching efficiency. Summary of the Invention

[0004] To solve the above-mentioned technical problems, or at least partially solve them, this disclosure provides an image matching method, apparatus, device, and medium.

[0005] According to one aspect of this disclosure, an image matching method is provided, comprising:

[0006] Get the image to be matched;

[0007] Extract the corresponding main feature points from the image to be matched according to the gradient direction angle, and determine the descriptor of the main feature points;

[0008] Obtain the target template image, and perform image matching between the image to be matched and the target template image based on the descriptor of the main feature points.

[0009] According to another aspect of this disclosure, an image matching apparatus is provided, comprising:

[0010] The acquisition module is used to acquire the image to be matched;

[0011] The extraction module is used to extract the corresponding main feature points of the image to be matched according to the gradient direction angle, and to determine the descriptor of the main feature points;

[0012] The matching module is used to acquire the target template image and perform image matching between the image to be matched and the target template image based on the descriptor of the main feature points.

[0013] According to another aspect of this disclosure, an electronic device is provided, comprising: a processor; and a memory storing a program, wherein the program includes instructions that, when executed by the processor, cause the processor to perform the image matching method described above.

[0014] According to another aspect of this disclosure, a computer-readable storage medium is provided, the storage medium storing a computer program for performing the above-described image matching method.

[0015] The image matching method and apparatus provided in this embodiment acquire an image to be matched; extract the corresponding main feature points of the image to be matched according to the gradient direction angle, and determine the descriptors of the main feature points; acquire a target template image, and perform image matching between the image to be matched and the target template image based on the descriptors of the main feature points. By employing the above scheme, extracting main feature points of the image to be matched according to the gradient direction and determining the descriptors of the main feature points, and performing image matching between the image to be matched and the target template image based on the descriptors of the main feature points, the main feature points have rich feature information. Even when the image to be matched is a weakly textured image, matching accuracy can be guaranteed. Furthermore, the complexity and resource consumption of image matching are reduced, matching efficiency is improved, and two images can be matched quickly.

[0016] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0017] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0018] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 A flowchart illustrating an image matching method provided in an embodiment of this disclosure;

[0020] Figure 2 A schematic diagram of a gradient direction provided in an embodiment of this disclosure;

[0021] Figure 3 A schematic diagram of a main feature point provided in an embodiment of this disclosure;

[0022] Figure 4A schematic diagram of a coarse descriptor provided for an embodiment of this disclosure;

[0023] Figure 5 A schematic diagram of a detailed descriptor provided in an embodiment of this disclosure;

[0024] Figure 6 A schematic flowchart of another image matching method provided in an embodiment of this disclosure;

[0025] Figure 7 A schematic diagram illustrating an image matching method provided in an embodiment of this disclosure;

[0026] Figure 8 This is a schematic diagram of the structure of an image matching device provided in an embodiment of the present disclosure;

[0027] Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0028] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0029] It should be understood that the steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.

[0030] The term "comprising" and its variations as used in this disclosure are open-ended, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below. It should be noted that the concepts of "first", "second", etc., mentioned in this disclosure are used only to distinguish different devices, modules, or units, and are not intended to limit the order of functions performed by these devices, modules, or units or their interdependencies.

[0031] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0032] To better understand the above-mentioned objectives, features, and advantages of this disclosure, the solutions disclosed herein will be further described below. It should be noted that, unless otherwise specified, the embodiments and features described herein can be combined with each other.

[0033] Image matching in related technologies can be performed using feature matching algorithms, template matching algorithms, or deep network algorithms. For example, feature matching algorithms can include SIFT, SURF, and Orb, but these cannot meet the alignment requirements of some weakly textured images. Deep network algorithms can include LIFT, SuperPoint, and SuperGlue, but they have high computational complexity, and template matching algorithms have high search complexity. Neither of these can meet the requirements of interactive response speed. In summary, related technologies have the following drawbacks: when the image is a weakly textured image, it is impossible to extract enough feature points, it cannot effectively describe the weakly textured image, and it cannot simultaneously meet the requirements of matching accuracy, complexity, resource consumption, and matching efficiency.

[0034] To improve at least one of the above problems, this disclosure provides an image matching method, apparatus, device, and medium, which are described below for ease of understanding.

[0035] Figure 1 This is a flowchart illustrating an image matching method provided in an embodiment of the present disclosure. The method can be executed by an image matching device, which can be implemented in software and / or hardware, and is generally integrated into an electronic device. Figure 1 As shown, the method mainly includes the following steps:

[0036] Step 101: Obtain the image to be matched.

[0037] The image matching method in this embodiment can be an important part of interactive scenarios. For example, the main process of paper screen interaction in a learning machine can include: subject area localization, identifier recognition, image matching, answer segmentation, grading strategy, and grading result. Image matching is an important part of this process. Paper screen interaction emphasizes strong real-time performance and strong stability. However, learning machine hardware resources are limited, and paper screen interaction is complex. Achieving image matching with low resource consumption can improve the effect of paper screen interaction.

[0038] The image to be matched can be one or more images that need to be matched. The image to be matched can be a real-time captured image, an image frame from a captured video, or an image input by the user. There are no specific limitations. For example, in a paper screen interaction scenario, the image to be matched can be a card containing learning content. This is just an example.

[0039] Step 102: Extract the corresponding main feature points of the image to be matched according to the gradient direction angle, and determine the descriptor of the main feature points.

[0040] The principal feature points can be feature points representing the main features of an image extracted using a gradient direction correlation algorithm. Since image edges can effectively describe weakly textured images, the principal feature points can be extracted from the image edges of the image to be processed, and there can be multiple principal feature points. The gradient can be used to measure the grayscale changes and level of an image's object contour. At each pixel in the image, the gradient has a gradient direction and a gradient magnitude. The gradient direction can have two parameters: gradient direction angle and gradient direction pointing. This scheme only considers the gradient direction angle and not the gradient direction pointing; that is, the gradient direction angle will take its absolute value. Therefore, the obtained gradient direction angle range is [0, 180°], avoiding the influence of the background on the gradient direction on both sides of the edge. A descriptor can describe the characteristics of a pixel within a certain area. The descriptor of the principal feature point can be information describing the characteristics of the principal feature point. In this embodiment, the descriptor can include two types: coarse descriptors and fine descriptors. The coarse descriptor can be a value obtained by analyzing and quantizing the gradient direction distribution of the area surrounding a point, while the fine descriptor can be a value obtained by analyzing and quantizing the relative gradient direction distribution of a pixel with other surrounding pixels.

[0041] In this embodiment of the present disclosure, the image matching device can extract the gradient direction angle of the image to be matched. The extraction method is not limited, such as using the Sobel operator, the Scharr operator, etc. Then, a gradient direction histogram can be constructed based on the gradient direction angle, and multiple main feature points can be determined based on the gradient direction histogram.

[0042] In some embodiments, extracting the corresponding main feature points of the image to be matched according to the gradient direction angle may include: dividing the image to be matched into multiple sub-regions; for each sub-region, determining the main feature direction based on the gradient direction histogram constructed based on the gradient direction angle of the sub-region, and determining the pixel point where the average coordinate of all pixels corresponding to the main feature direction is located as the main feature point corresponding to the sub-region.

[0043] A sub-region can be a small area obtained by dividing an image into equal parts. The number of sub-regions is unlimited and can be set according to the actual situation. A Histogram of Oriented Gradient (HOG) is a method for visualizing the distribution of gradient directions in an image. It visualizes the gradient distribution by calculating the gradient at each pixel in a local region of the image and classifying it into a histogram. The principal feature direction can be the dominant direction determined for each sub-region based on the gradient direction distribution. Each sub-region has a corresponding principal feature direction and principal feature point.

[0044] Specifically, when extracting the corresponding principal feature points from the image to be matched, the image matching device can first extract the gradient direction angle of the image to be matched, divide the image to be matched into multiple sub-regions, and construct a gradient direction histogram for each sub-region according to the gradient direction angle. The specific construction process can include using the gradient direction angle as the horizontal axis and the number of pixels as the vertical axis, dividing the gradient direction angle range of 180° into multiple groups, such as 8 or 9 groups, each group being 22.5° or 20°. The pixels of each sub-region are assigned to each group according to the gradient direction angle, and the number of pixels included in each group is used as the value of the corresponding rectangular block. For example, if the number of pixels in the group with a gradient direction angle of 0°-20° is 5, then the value of the corresponding rectangular block for this group is 5, thus obtaining the gradient direction histogram for each sub-region. Then, for each sub-region, the direction of the gradient direction angle range of the rectangular block where the peak value of the corresponding gradient direction histogram is located is determined as the principal feature direction of the sub-region. The coordinate mean of all pixels corresponding to the principal feature direction is calculated, and the pixel where the coordinate mean is located is determined as the principal feature point corresponding to the sub-region.

[0045] For example, Figure 2 A schematic diagram of a gradient direction provided in an embodiment of this disclosure, such as... Figure 2 As shown in the figure, the gradient direction angles v1 and v2 of a pixel in the image to be matched under different backgrounds are displayed. The angle between the two is 180 degrees. When v2 is mapped to v′2, the gradient direction angles of v′2 and v1 are the same. That is, when the angle between the two gradient direction angles is 180 degrees, the same gradient direction angle is obtained through mapping. It can be seen that after mapping, the gradient direction of the image edge is not affected by the background on both sides of the edge. Therefore, the gradient direction angle of this scheme will take the absolute value. Thus, the range of the gradient direction angle obtained is [0, 180°].

[0046] For example, Figure 3 A schematic diagram of a main feature point provided in an embodiment of this disclosure, such as... Figure 3 As shown in the figure, the figure illustrates the main feature points of an image to be matched, with the black dots in the figure representing the main feature points.

[0047] In some embodiments, determining the descriptor of the main feature point may include: quantizing the gradient direction histogram of the rectangular region corresponding to the main feature point of the image to be matched based on a preset threshold to obtain the corresponding coarse descriptor; extracting multiple uniformly distributed pixels in the rectangular region corresponding to the main feature point of the image to be matched, and quantizing the relative gradient direction angle of the multiple uniformly distributed pixels with respect to the main feature point of the image to be matched to obtain the corresponding fine descriptor.

[0048] The preset threshold can be a numerical threshold for the number of pixels, which can be set according to actual conditions; for example, the preset threshold can be set to 10 or 20. The rectangular region corresponding to the main feature point can be a rectangular region with the main feature point as its center and a preset size. The preset size can be set according to actual conditions; for example, the preset size can be 2*2 or 3*3. Uniformly distributed pixels can be pixels within the rectangular region corresponding to the main feature point that are within the same angular range as the main direction of the rectangular region. The main direction of the rectangular region is the angular range of the gradient direction angle of the rectangular block containing the peak value in the gradient direction histogram of the corresponding rectangular region.

[0049] When determining the descriptor of the main feature point of the image to be matched, the image matching device can first extract the gradient of all pixels in the rectangular region where the main feature point is located, and remove pixels whose gradient magnitude is less than the gradient threshold. For the coarse descriptor, a gradient direction histogram is constructed based on the gradient direction angle of the remaining pixels. The construction method is the same as in the above embodiment. The number of pixels included in the angle range of each rectangular block of the gradient direction histogram is judged to be greater than a preset threshold. If it is, the quantization value of the angle range of the rectangular block is 1; otherwise, the quantization value of the angle range of the rectangular block is 0. Thus, the gradient direction histogram is quantized to include only 0 and 1 quantization values, and this quantization value is the coarse descriptor.

[0050] For example, Figure 4 This is a schematic diagram of a coarse descriptor provided in an embodiment of the present disclosure, such as... Figure 4 As shown in the figure, the gradient direction distribution corresponding to a main feature point in the image to be matched is displayed. It can be seen that the angle range of 67.5°-90° contains the most pixels. Therefore, the direction corresponding to this angle range is the main direction of the rectangular region corresponding to the main feature point. Furthermore, each angle range is quantized by comparing the number of pixels in each angle range with a preset threshold. As shown in the figure, only the number of pixels in the angle range of 67.5°-90° is greater than the preset threshold, corresponding to a quantization value of 1. The number of pixels in other angle ranges is less than the preset threshold, corresponding to a quantization value of 0. Therefore, the coarse descriptor is 00010000.

[0051] For fine descriptors, multiple uniformly distributed pixels can be extracted from the rectangular region corresponding to the main feature point, and the relative gradient direction angle of each uniformly distributed pixel relative to the main feature point can be determined. The relative gradient direction angles of multiple uniformly distributed pixels relative to the main feature point are quantized separately. When quantizing a relative gradient direction angle, the quantization value of the angle range in which the relative gradient direction angle is located is 1, and the quantization value of other angle ranges is 0, resulting in an 8-bit quantization value. The multiple quantization values ​​of multiple uniformly distributed pixels are determined as fine descriptors.

[0052] For example, Figure 5A schematic diagram of a detailed descriptor provided in an embodiment of this disclosure, such as... Figure 5 As shown, the center point in the figure is a main feature point of the image to be matched. The main feature point is surrounded by multiple uniformly distributed pixels. Each uniformly distributed pixel has an 8-bit quantization value. The combination of multiple 8-bit quantization values ​​determines the fine descriptor of the main feature point.

[0053] It is understood that the embodiments of this disclosure have described the process of determining the coarse descriptor and fine descriptor of a pixel in an image by taking the determination of the main feature points of the image to be matched as an example. The coarse descriptor and fine descriptor of other pixels in the image can also be determined in the same way, and will not be described in detail.

[0054] Step 103: Obtain the target template image and perform image matching between the image to be matched and the target template image based on the descriptor of the main feature points.

[0055] The target template image can be a standard image that is matched or aligned with the image to be matched mentioned above. It can be obtained from a preset template library. For example, in a paper screen interactive scenario, the target template image can be a pre-set standard image of a learning card.

[0056] The image matching device can acquire a target template image and determine the corresponding main feature points for the target template image using the method described in step 102 above. Then, it performs image matching on the two images based on the main feature points corresponding to the target template image and the image to be matched, respectively.

[0057] For example, Figure 6 A flowchart illustrating another image matching method provided in this disclosure embodiment is shown below. Figure 6 As shown, in one feasible implementation, image matching is performed between the image to be matched and the target template image based on the descriptor of the main feature points, including:

[0058] Step 601: Extract the main feature points of the target template image and determine the main matching point based on the main feature points.

[0059] The primary matching point can be the center point of a region in an image where primary feature points are relatively concentrated. The primary matching point is related to the primary feature points and works with them to align the pixels of the two images. There can be multiple primary matching points.

[0060] The image matching device determines the corresponding main feature point for the target template image using the method described in step 102 above. The difference is that the size of the sub-region divided for the target template image can be larger than the size of the sub-region divided for the image to be matched, to ensure subsequent matching. Then, the main matching point is determined based on the main feature point.

[0061] In some embodiments, determining the main matching point based on the main feature points may include: dividing the target template image into multiple grids; identifying the target grid in the multiple grids where the number of main feature points is greater than a threshold, and determining the pixel point corresponding to the average coordinate of all main feature points in the target grid as the main matching point.

[0062] The grid can be a unit obtained by meshing the target template image. The number of grids can be multiple, depending on the specific situation. For example, the target template image can be divided into multiple 10*10 grids. The target grid can be a grid in the multiple grids of the target template image where the number of principal feature points is greater than a certain threshold. The threshold can be set according to the specific situation, for example, the threshold can be 20.

[0063] Specifically, when determining the main matching point based on the main feature points, the image matching device can first perform gridding processing on the target template image to obtain multiple grids. It can then determine whether the number of main feature points included in each grid is greater than the number threshold. If so, the grid is the target grid. The average coordinate value of all main feature points included in the target grid is calculated, and the pixel point corresponding to the average coordinate value is determined as the main matching point. There are multiple target grids and multiple main matching points.

[0064] Step 602: Determine the main feature point corresponding to the main matching point of the target template image in the image to be matched as the first feature point, and based on the relative positional relationship between the main feature point and the main matching point of the target template image and the position of the first feature point, determine the second feature point corresponding to the main feature point of the target template image in the image to be matched.

[0065] The first feature point can be a point in the image to be matched where each main matching point in the target template image is located on a main feature point in the image to be matched. The relative positional relationship between the main feature point and the main matching point in the target template image can be either the relative coordinates of the main feature point relative to the main matching point or the relative coordinates of the main matching point relative to the main feature point. The second feature point can be a pixel in the image to be matched that corresponds to a main feature point in the target template image.

[0066] The image matching device can locate each main matching point of the target template image onto a main feature point of the image to be matched, thereby obtaining the first feature point. Then, based on the position of the first feature point, it determines a pixel in the image to be matched whose relative positional relationship is the same as that of the main feature point and the main matching point of the target template image. This pixel is the second feature point corresponding to the main feature point of the target template image in the image to be matched.

[0067] Step 603: Based on the descriptors of the main feature points and main matching points of the target template image and the descriptors of the first feature points and second feature points in the image to be matched, determine the matching points in the image to be matched that match the main feature points and main matching points of the target template image, respectively.

[0068] The image matching device can determine the main feature points and descriptors of the main matching points of the target template image, and determine the descriptor of the second feature point in the image to be matched. The method of determining the descriptor is the same as in the above embodiment, and will not be repeated here. The first feature point is the main feature point of the image to be matched, and its descriptor has been obtained in step 102. Then, based on the descriptors of the main feature points and main matching points of the target template image and the descriptors of the first feature points and second feature points in the image to be matched, image matching is performed on the target template image and the image to be matched to determine the matching points in the image to be matched that match the main feature points and main matching points of the target template image, respectively.

[0069] In some embodiments, the matching points include a first matching point and a second matching point. Based on the main feature points of the target template image, the descriptor of the main matching point, and the descriptors of the first and second feature points in the image to be matched, determining the matching points in the image to be matched that match the main feature points and the main matching point of the target template image, respectively, may include: performing a coarse match between the image to be matched and the target template image based on the coarse descriptor of the main feature points of the target template image and the coarse descriptor of the second feature points in the image to be matched, to determine the coarse matching points in the image to be matched that match the main matching points of the target template image; determining the fine descriptor of the main matching points of the target template image, the fine descriptors of each position point in the preset region where the second feature points are located in the image to be matched, and the fine descriptor of the coarse matching points in the image to be matched; and performing a fine match between the image to be matched and the target template image based on the main feature points of the target template image, the fine descriptors of the main matching points, the fine descriptors of each position point in the preset region where the second feature points are located in the image to be matched, and the fine descriptors of the coarse matching points in the image to be matched, to determine the first matching point in the image to be matched that matches the main feature points of the target template image, and the second matching point in the image to be matched that matches the main matching points of the target template image among the coarse matching points.

[0070] This disclosure embodiment allows for two stages in matching two images: coarse matching and fine matching. Coarse matching analyzes the gradient direction distribution around the main feature points to determine potential correspondences between different regions. Fine matching further identifies these specific correspondences. Coarse matching quickly locates potential matching points, allowing for more complex operations during precise matching without affecting the algorithm's real-time performance. This disclosure embodiment uses different descriptors for different matching processes; coarse matching is achieved using a coarse descriptor, while fine matching is achieved using a fine descriptor.

[0071] Optionally, based on the coarse descriptor of the main feature point of the target template image and the coarse descriptor of the second feature point in the image to be matched, a coarse matching is performed between the image to be matched and the target template image to determine the coarse matching point in the image to be matched that matches the main matching point of the target template image. This can include: based on the coarse descriptor of the main feature point corresponding to the main matching point of the target template image and the coarse descriptor of the second feature point in the image to be matched, determining the similarity between the main matching point of the target template image and the first feature point corresponding to the main matching point in the image to be matched through similarity calculation; and determining the first feature point in the image to be matched with a similarity greater than a similarity threshold as the coarse matching point.

[0072] The coarse matching point can be a first feature point in the image to be matched that matches the main matching point of the target template image. The first feature point is the main feature point of the image to be matched. In other words, the coarse matching process matches the main feature point of the image to be matched with the main matching point of the target template image.

[0073] When performing coarse matching between a target template image and an image to be matched, the image matching device can first determine the similarity between the main matching point of the target template image and the first feature point located by the main matching point in the image to be matched. Specifically, the similarity can be calculated based on the coarse descriptors of multiple main feature points corresponding to the main matching point of the target template image and multiple second feature points corresponding to the first feature point. The multiple main feature points corresponding to the main matching point are the multiple main feature points within the target grid corresponding to the main matching point during the determination process. Then, the first feature point with a similarity greater than the similarity threshold can be determined as the coarse matching point. The similarity threshold can be set according to the actual situation.

[0074] Input the coarse descriptors of multiple principal feature points corresponding to the principal matching points in the target template image, and the coarse descriptors of multiple second feature points corresponding to the first feature point in the image to be matched, into the formula. In, among which, s c The similarity between the main matching point of the target template image and a certain first feature point in the image to be matched is calculated. This is a logical AND operation, where k is the number of principal feature points corresponding to the principal matching points in the target template image. This is the coarse descriptor corresponding to the principal feature point of the principal matching point in the target template image. onecount(A) represents the number of 1s in the value A. This is a coarse description of multiple second feature points corresponding to the first feature point in the image to be matched. The second feature points are obtained through the relative positional relationship between the main matching point and the main feature point in the target template image. The relative positional relationship between the main matching point and the main feature point in the target template image is the same as the relative positional relationship between the first feature point and the second feature point in the image to be matched.

[0075] Optionally, based on the main feature points of the target template image, the fine descriptor of the main matching point, the fine descriptor of each position point in the preset region where the second feature point is located in the image to be matched, and the fine descriptor of the coarse matching point in the image to be matched, a fine matching is performed between the image to be matched and the target template image to determine the first matching point in the image to be matched that matches the main feature point of the target template image, and the second matching point in the coarse matching point of the image to be matched that matches the main matching point of the target template image. This can include: based on the fine descriptor of the main feature points of the target template image and the fine descriptor of each pixel point in the preset region where the second feature point is located in the image to be matched. The similarity calculation determines the similarity between the main feature point of the target template image and each pixel in the preset region where the second feature point in the image to be matched is located. The pixel with the highest similarity in the preset region where the second feature point in the image to be matched is determined as the first matching point that matches the main feature point of the target template image. Based on the fine descriptor of the main matching point of the target template image and the fine descriptor of each coarse matching point in the image to be matched, the similarity between the main matching point of the target template image and each coarse matching point in the image to be matched is determined, and the coarse matching point with the highest similarity is determined as the second matching point that matches the main matching point of the target template image.

[0076] The preset area where the second feature point is located can be a small area centered on the second feature point. The specific size can be set according to the actual situation. For example, the size of the preset area can be 3*3.

[0077] After coarse matching of the target template image and the image to be matched, the image matching device determines that the main matching points of the target template image correspond to the coarse matching points of the image to be matched, and the main feature points of the target template image correspond to the second feature points of the image to be matched. Based on this, fine matching continues. Specifically, the similarity between the main feature points of the target template image and the pixels in the preset region containing the second feature points of the image to be matched is determined through similarity calculation. The specific determination method involves inputting the fine descriptors of the main feature points of the target template image and the fine descriptors of the pixels in the preset region containing the second feature points of the image to be matched into a formula. In the middle, s r This indicates the similarity between the main feature points of the target template image and a pixel in a preset region where the second feature point of the image to be matched is located. For logical XOR operation, d tj A fine descriptor representing the principal feature points of the target template image, d qj This represents the fine descriptor of a pixel in the preset region where the second feature point is located on the image to be matched; then, the pixel with the highest similarity in the preset region where the second feature point is located can be determined as the first matching point that matches the main feature point of the target template image.

[0078] The image matching device determines the similarity between the main matching points of the target template image and each coarse matching point in the image to be matched by similarity calculation. Specifically, the fine descriptors of the main matching points of the target template image and the fine descriptors of each coarse matching point in the image to be matched are input into a formula. In the middle, s q d represents the similarity between the main matching point in the target template image and a coarse matching point in the image to be matched. tk d represents the fine descriptor of the main matching points of the target template image. qk This represents a fine description of a coarse matching point in the image to be matched. This is a logical XOR operation; afterwards, the coarse matching point with the highest similarity can be determined as the second matching point that matches the main matching point of the target template image.

[0079] This scheme provides two novel feature descriptors: a coarse descriptor and a precise descriptor. Both have good descriptive capabilities for weakly textured images. Combined with appropriate search strategies, they can achieve rapid alignment of weakly textured images. Furthermore, it offers coarse matching and fine matching. Coarse matching significantly improves the algorithm's matching speed, while fine matching, while ensuring real-time performance, enhances the algorithm's robustness to image distortion and can quickly determine the pixel-level alignment relationship between the target template image and the image to be matched.

[0080] In summary, the image matching method provided in this disclosure involves: acquiring an image to be matched; extracting corresponding main feature points from the image to be matched according to the gradient direction angle and determining the descriptors of the main feature points; acquiring a target template image; and performing image matching between the image to be matched and the target template image based on the descriptors of the main feature points. By employing this scheme, extracting main feature points from the image to be matched according to the gradient direction and determining the descriptors of these main feature points, and then performing image matching between the image to be matched and the target template image based on these descriptors, the main feature points provide rich feature information. This ensures matching accuracy even when the image to be matched is a weakly textured image, reduces the complexity and resource consumption of image matching, improves matching efficiency, and enables rapid matching of two images.

[0081] In some embodiments, obtaining a target template image may include: determining a first principal direction based on the principal feature points of the image to be matched to obtain multiple candidate template images, and determining multiple second principal directions corresponding to the multiple candidate template images; determining the candidate template image whose second principal direction is the same as the first principal direction of the image to be matched as the target template image. Optionally, determining the corresponding first principal direction based on the principal feature points of the image to be matched may include: constructing a gradient direction histogram based on the gradient direction angles of the principal feature points of the image to be matched, and determining the direction with the most principal feature points in the gradient direction histogram as the first principal direction.

[0082] The first principal direction can be the principal direction of the image to be matched, determined based on the gradient direction angle of the principal feature points. The second principal direction can be the principal direction of the candidate template image, determined based on the gradient direction angle of the principal feature points. The candidate template images can be template images from a template library, and the specific number is unlimited. The target template image is a candidate template image determined from multiple candidate template images for image matching or image alignment with the image to be matched.

[0083] The image matching device first determines the first principal direction of the image to be matched based on the principal feature points. Specifically, a gradient direction histogram can be constructed based on the gradient direction angles of the principal feature points. The gradient direction angles can be used as the horizontal axis, and the number of principal feature points as the vertical axis. The gradient direction angle range of 180° is divided into multiple groups, such as 8 or 9 groups, each group being 22.5° or 20°. The principal feature points of the image to be matched are assigned to the angle range of each group according to the gradient direction angle. The number of principal feature points included in each group is used as the value of the corresponding rectangular block, resulting in a gradient direction histogram. The direction of the gradient direction angle range with the highest number of principal feature points in the gradient direction histogram is determined as the first principal direction of the image to be matched. The second principal direction of each candidate template image can also be determined in the same way. Then, the second principal direction of each candidate template image is compared with the first principal direction of the image to be matched. The candidate template image whose second principal direction is the same as the first principal direction of the image to be matched is determined as the target template image.

[0084] When a strong robustness to changes in viewpoint is required, it is necessary to match a large number of template images from different viewpoints. The increase in the number of template images leads to a surge in the number of main matching points and main feature points, severely impacting the algorithm's real-time performance. In the above scheme, the main directions of the image to be matched and the candidate template images can be extracted using the gradient direction angle of the main feature points. Then, candidate template images with the same main direction are selected as the target template image for the current matching, avoiding image matching with a large number of template images and improving the real-time performance of subsequent image matching through template selection.

[0085] In some embodiments of this disclosure, after obtaining the image to be matched in step 101 above, the image matching method may further include: preprocessing the image to be matched, which may include at least one of scaling, smoothing, and grayscale conversion.

[0086] The scaling mentioned above can be to scale the image to be matched to a low-resolution three-channel image, such as an image with a width of 640, a height of 480, and 3 channels. By reducing the resolution, the subsequent computational load can be reduced. The grayscale conversion mentioned above can be to convert the image to be matched to a single-channel grayscale image. Since image matching is not sensitive to the number of channels, converting to grayscale data can further reduce the subsequent computational load. The smoothing mentioned above can be to apply Gaussian smoothing to the image to be matched using a 3x3 convolution kernel. Smoothing reduces the influence of the environment, lowers the proportion of noise, and improves the robustness of the image processing method.

[0087] In scenarios such as paper-screen interaction, the raw images captured by the camera can be three-channel high-resolution image frames, and processing these three-channel high-resolution image frames is time-consuming. The solution disclosed above reduces the complexity of subsequent image processing by performing image preprocessing operations such as scaling, smoothing, and grayscale conversion on the images to be matched.

[0088] For example, Figure 7 This is a schematic diagram of an image matching embodiment provided in this disclosure, such as... Figure 7 As shown in the figure, the complete process of image matching can be included, specifically: startup, preprocessing, extraction of main feature points, extraction of coarse descriptors, extraction of fine descriptors, coarse matching, fine matching, and matching optimization.

[0089] Corresponding to the aforementioned image matching method, this disclosure also provides an image matching apparatus. Figure 8 This is a schematic diagram of the structure of an image matching device provided in an embodiment of the present disclosure. This device can be implemented by software and / or hardware, and is generally integrated into an electronic device. Figure 8 As shown, the image matching device 800 includes:

[0090] The acquisition module 801 is used to acquire the image to be matched;

[0091] Extraction module 802 is used to extract the corresponding main feature points of the image to be matched according to the gradient direction angle, and determine the descriptor of the main feature points;

[0092] The matching module 803 is used to acquire the target template image and perform image matching between the image to be matched and the target template image based on the descriptor of the main feature points.

[0093] In the above device, the main feature points are extracted from the image to be matched according to the gradient direction and the descriptor of the main feature points is determined. Based on the descriptor of the main feature points, image matching is performed between the image to be matched and the target template image. The main feature points have rich feature information, which can ensure matching accuracy even when the image to be matched is a weak texture image. It also reduces the complexity and resource consumption of image matching, improves matching efficiency, and can quickly match two images.

[0094] In some implementations, the extraction module 802 includes feature point units for:

[0095] The image to be matched is divided into multiple sub-regions;

[0096] For each sub-region, the main feature direction is determined by the gradient direction histogram constructed based on the gradient direction angle of the sub-region, and the pixel point where the average coordinate of all pixels corresponding to the main feature direction is located is determined as the main feature point corresponding to the sub-region.

[0097] In some embodiments, the descriptor includes a coarse descriptor and a fine descriptor, and the extraction module 802 includes a descriptor unit for:

[0098] Based on a preset threshold, the gradient direction histogram of the rectangular region corresponding to the main feature point of the image to be matched is quantized to obtain the corresponding coarse descriptor;

[0099] Multiple uniformly distributed pixels are extracted from the rectangular region corresponding to the main feature point of the image to be matched, and the relative gradient direction angle of the multiple uniformly distributed pixels with respect to the main feature point of the image to be matched is quantized to obtain the corresponding fine descriptor.

[0100] In some implementations, the matching module 803 includes:

[0101] The first unit is used to extract the main feature points of the target template image and determine the main matching point based on the main feature points;

[0102] The second unit is used to determine the main feature point corresponding to the main matching point of the target template image in the image to be matched as the first feature point, and to determine the second feature point corresponding to the main feature point of the target template image in the image to be matched based on the relative positional relationship between the main feature point and the main matching point of the target template image and the position of the first feature point;

[0103] The third unit is used to determine, based on the descriptors of the main feature points and main matching points of the target template image and the descriptors of the first feature points and second feature points in the image to be matched, the matching points in the image to be matched that match the main feature points and main matching points of the target template image, respectively.

[0104] In some embodiments, the matching point includes a first matching point and a second matching point, and the third unit includes:

[0105] The coarse matching subunit is used to perform coarse matching on the image to be matched and the target template image based on the coarse descriptor of the main feature point of the target template image and the coarse descriptor of the second feature point in the image to be matched, and to determine the coarse matching point in the image to be matched that matches the main matching point of the target template image.

[0106] The fine matching subunit is used to perform fine matching on the image to be matched and the target template image based on the main feature points of the target template image, the fine descriptors of the main matching points, the fine descriptors of each position point in the preset region where the second feature point in the image to be matched is located, and the fine descriptors of the coarse matching points in the image to be matched, to determine the first matching point in the image to be matched that matches the main feature points of the target template image, and the second matching point in the coarse matching points of the image to be matched that matches the main matching points of the target template image.

[0107] In some implementations, the coarse matching subunit is used for:

[0108] Based on the coarse descriptor of the main feature point corresponding to the main matching point of the target template image and the coarse descriptor of the second feature point in the image to be matched, the similarity between the main matching point of the target template image and the first feature point corresponding to the main matching point in the image to be matched is determined by similarity calculation.

[0109] The first feature point in the image to be matched with a similarity greater than the similarity threshold is determined as the coarse matching point.

[0110] In some implementations, the fine-matching subunit is used for:

[0111] Based on the detailed descriptor of the main feature point of the target template image and the detailed descriptor of each pixel in the preset region where the second feature point is located in the image to be matched, the similarity between the main feature point of the target template image and each pixel in the preset region where the second feature point is located in the image to be matched is determined by similarity calculation.

[0112] The pixel with the highest similarity in the preset region where the second feature point is located in the image to be matched is determined as the first matching point that matches the main feature point of the target template image;

[0113] Based on the fine descriptor of the main matching point of the target template image and the fine descriptor of each coarse matching point in the image to be matched, the similarity between the main matching point of the target template image and each coarse matching point in the image to be matched is determined, and the coarse matching point with the highest similarity is determined as the second matching point that matches the main matching point of the target template image.

[0114] In some embodiments, the first unit is used for:

[0115] The target template image is divided into multiple grids;

[0116] A target grid with a number of main feature points greater than a threshold is identified among the multiple grids, and the pixel corresponding to the average coordinate of all main feature points in the target grid is determined as the main matching point.

[0117] In some embodiments, the matching module 803 includes an acquisition unit for:

[0118] The first principal direction is determined based on the main feature points of the image to be matched.

[0119] Acquire multiple candidate template images and determine multiple second principal directions corresponding to the multiple candidate template images;

[0120] The candidate template image whose second principal direction is the same as the first principal direction of the image to be matched is determined as the target template image.

[0121] In some embodiments, the acquisition unit is specifically used for:

[0122] A gradient direction histogram is constructed based on the gradient direction angle of the main feature points of the image to be matched, and the direction with the most main feature points in the gradient direction histogram is determined as the first main direction.

[0123] The image matching apparatus provided in this disclosure can execute the image matching method provided in any embodiment of this disclosure, and has the corresponding functional modules and beneficial effects of executing the method.

[0124] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the above-described device embodiments can be referred to the corresponding process in the method embodiments, and will not be repeated here.

[0125] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0126] Exemplary embodiments of this disclosure also provide an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor. The memory stores a computer program executable by the at least one processor, the computer program being executed by the at least one processor to cause the electronic device to perform a method according to an embodiment of this disclosure.

[0127] Exemplary embodiments of this disclosure also provide a non-transitory computer-readable storage medium storing a computer program, wherein the computer program, when executed by a computer's processor, is used to cause the computer to perform a method according to embodiments of this disclosure.

[0128] Exemplary embodiments of this disclosure also provide a computer program product, including a computer program, wherein, when executed by a processor of a computer, the computer program is used to cause the computer to perform a method according to an embodiment of this disclosure.

[0129] The computer program product can be written in any combination of one or more programming languages ​​to perform the operations of the embodiments of this disclosure. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on a user's computing device, partially on a user's computing device, as a standalone software package, partially on a user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0130] Furthermore, embodiments of this disclosure can also be computer-readable storage media storing computer program instructions that, when executed by a processor, cause the processor to perform the XYZ method provided in embodiments of this disclosure. The computer-readable storage medium can be any combination of one or more readable media. A readable medium can be a readable signal medium or a readable storage medium. A readable storage medium can be, for example, including but not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0131] Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. (Reference) Figure 9The present invention describes a structural block diagram of an electronic device 900 that can serve as a server or client of the present disclosure, which is an example of a hardware device that can be applied to various aspects of the present disclosure. The electronic device is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0132] like Figure 9 As shown, the electronic device 900 includes a computing unit 901, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 902 or a computer program loaded from a storage unit 908 into a random access memory (RAM) 903. The RAM 903 may also store various programs and data required for the operation of the device 900. The computing unit 901, ROM 902, and RAM 903 are interconnected via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0133] Multiple components in electronic device 900 are connected to I / O interface 905, including: input unit 906, output unit 907, storage unit 908, and communication unit 909. Input unit 906 can be any type of device capable of inputting information to electronic device 900. Input unit 906 can receive input digital or character information and generate key signal inputs related to user settings and / or function control of electronic device. Output unit 907 can be any type of device capable of presenting information and may include, but is not limited to, a display, speaker, video / audio output terminal, vibrator, and / or printer. Storage unit 908 may include, but is not limited to, disk and optical disk. Communication unit 909 allows electronic device 900 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks, and may include, but is not limited to, modems, network cards, infrared communication devices, wireless communication transceivers and / or chipsets, such as Bluetooth™ devices, WiFi devices, WiMax devices, cellular communication devices, and / or the like.

[0134] The computing unit 901 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 901 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 901 performs the various methods and processes described above. For example, in some embodiments, the image matching methods can be implemented as computer software programs tangibly contained in a machine-readable medium, such as storage unit 908. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 900 via ROM 902 and / or communication unit 909. In some embodiments, the computing unit 901 can be configured to perform the image matching methods by any other suitable means (e.g., by means of firmware).

[0135] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0136] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0137] As used in this disclosure, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, device, and / or apparatus (e.g., disk, optical disk, memory, programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including machine-readable media that receive machine instructions as machine-readable signals. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.

[0138] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0139] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0140] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other.

[0141] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0142] The above description is merely a specific embodiment of this disclosure, enabling those skilled in the art to understand or implement it. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this disclosure. Therefore, this disclosure is not to be limited to the embodiments described herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An image matching method, comprising: Get the image to be matched; Extract the corresponding main feature points from the image to be matched according to the gradient direction angle, and determine the descriptor of the main feature points; Obtain the target template image, and perform image matching between the image to be matched and the target template image based on the descriptor of the main feature points; The image matching between the image to be matched and the target template image based on the descriptor of the main feature points includes: Extract the main feature points of the target template image, and determine the main matching point based on the main feature points; The main feature point corresponding to the main matching point of the target template image in the image to be matched is determined as the first feature point. Based on the relative positional relationship between the main feature point and the main matching point of the target template image and the position of the first feature point, the second feature point corresponding to the main feature point of the target template image in the image to be matched is determined.

2. The method as described in claim 1, wherein, The step of extracting the corresponding main feature points of the image to be matched according to the gradient direction angle includes: The image to be matched is divided into multiple sub-regions; For each sub-region, the main feature direction is determined by the gradient direction histogram constructed based on the gradient direction angle of the sub-region, and the pixel point where the average coordinate of all pixels corresponding to the main feature direction is located is determined as the main feature point corresponding to the sub-region.

3. The method as described in claim 1, wherein, The descriptor includes a coarse descriptor and a fine descriptor. The descriptor that determines the principal feature point includes: Based on a preset threshold, the gradient direction histogram of the rectangular region corresponding to the main feature point of the image to be matched is quantized to obtain the corresponding coarse descriptor; Multiple uniformly distributed pixels are extracted from the rectangular region corresponding to the main feature point of the image to be matched, and the relative gradient direction angle of the multiple uniformly distributed pixels with respect to the main feature point of the image to be matched is quantized to obtain the corresponding fine descriptor.

4. The method of claim 1, wherein, After determining the main feature point of the target template image and the corresponding second feature point in the image to be matched, the method further includes: Based on the descriptors of the main feature points and main matching points of the target template image and the descriptors of the first feature points and second feature points in the image to be matched, the matching points in the image to be matched that match the main feature points and main matching points of the target template image are determined respectively.

5. The method of claim 4, wherein, The matching points include a first matching point and a second matching point. Based on the descriptors of the main feature points and main matching points of the target template image and the descriptors of the first feature points and second feature points in the image to be matched, matching points in the image to be matched that match the main feature points and main matching points of the target template image are determined, including: Based on the coarse descriptor of the main feature points of the target template image and the coarse descriptor of the second feature points in the image to be matched, a coarse match is performed between the image to be matched and the target template image to determine the coarse matching points in the image to be matched that match the main matching points of the target template image. Based on the main feature points and the fine descriptor of the main matching point of the target template image, the fine descriptor of each position point in the preset region where the second feature point in the image to be matched is located, and the fine descriptor of the coarse matching point in the image to be matched, a fine matching is performed on the image to be matched and the target template image to determine the first matching point in the image to be matched that matches the main feature point of the target template image, and the second matching point in the coarse matching point of the image to be matched that matches the main matching point of the target template image.

6. The method of claim 5, wherein, Based on the coarse descriptor of the main feature points of the target template image and the coarse descriptor of the second feature points in the image to be matched, a coarse match is performed between the image to be matched and the target template image to determine the coarse matching points in the image to be matched that match the main matching points of the target template image, including: Based on the coarse descriptor of the main feature point corresponding to the main matching point of the target template image and the coarse descriptor of the second feature point in the image to be matched, the similarity between the main matching point of the target template image and the first feature point corresponding to the main matching point in the image to be matched is determined by similarity calculation. The first feature point in the image to be matched with a similarity greater than the similarity threshold is determined as the coarse matching point.

7. The method of claim 5, wherein, Based on the main feature points and the fine descriptor of the main matching point of the target template image, the fine descriptor of each position point in the preset region where the second feature point in the image to be matched is located, and the fine descriptor of the coarse matching point in the image to be matched, a fine matching is performed on the image to be matched and the target template image to determine the first matching point in the image to be matched that matches the main feature point of the target template image, and the second matching point in the coarse matching point of the image to be matched that matches the main matching point of the target template image, including: Based on the detailed descriptor of the main feature point of the target template image and the detailed descriptor of each pixel in the preset region where the second feature point is located in the image to be matched, the similarity between the main feature point of the target template image and each pixel in the preset region where the second feature point is located in the image to be matched is determined by similarity calculation; The pixel with the highest similarity in the preset region where the second feature point is located in the image to be matched is determined as the first matching point that matches the main feature point of the target template image; Based on the fine descriptor of the main matching point of the target template image and the fine descriptor of each coarse matching point in the image to be matched, the similarity between the main matching point of the target template image and each coarse matching point in the image to be matched is determined, and the coarse matching point with the highest similarity is determined as the second matching point that matches the main matching point of the target template image.

8. The method of claim 4, wherein, The process of determining the main matching point based on the main feature point includes: The target template image is divided into multiple grids; A target grid with a number of main feature points greater than a threshold is identified among the multiple grids, and the pixel corresponding to the average coordinate of all main feature points in the target grid is determined as the main matching point.

9. The method of claim 1, wherein, The acquisition of the target template image includes: The first principal direction is determined based on the main feature points of the image to be matched. Acquire multiple candidate template images and determine multiple second principal directions corresponding to the multiple candidate template images; The candidate template image whose second principal direction is the same as the first principal direction of the image to be matched is determined as the target template image.

10. The method of claim 9, wherein, Determining the corresponding first principal direction based on the principal feature points of the image to be matched includes: A gradient direction histogram is constructed based on the gradient direction angle of the main feature points of the image to be matched, and the direction with the most main feature points in the gradient direction histogram is determined as the first main direction.

11. An image matching device, comprising: The acquisition module is used to acquire the image to be matched; The extraction module is used to extract the corresponding main feature points of the image to be matched according to the gradient direction angle, and to determine the descriptor of the main feature points; The matching module is used to acquire the target template image and perform image matching between the image to be matched and the target template image based on the descriptor of the main feature points; The image matching between the image to be matched and the target template image based on the descriptor of the main feature points includes: Extract the main feature points of the target template image, and determine the main matching point based on the main feature points; The main feature point corresponding to the main matching point of the target template image in the image to be matched is determined as the first feature point. Based on the relative positional relationship between the main feature point and the main matching point of the target template image and the position of the first feature point, the second feature point corresponding to the main feature point of the target template image in the image to be matched is determined.

12. An electronic device, comprising: processor; as well as Stored program memory, The program includes instructions that, when executed by the processor, cause the processor to perform the image matching method according to any one of claims 1-10.

13. A computer-readable storage medium storing a computer program for performing the image matching method according to any one of claims 1-10.

Images