Image matching positioning kernel region automatic extraction method, device, equipment and medium

Abstract

The application relates to a positioning core region automatic extraction method, device and equipment and a medium. The method comprises the following steps: acquiring a set reference image and image matching data; determining a candidate region of the reference image according to the image matching data; and extracting a positioning core region in the candidate region, which comprises the following steps: performing feature extraction on the candidate region to generate a feature point set; determining a plurality of minimum circumscribed positive rectangles according to the feature point set to obtain a plurality of rectangular regions; performing uniqueness detection on the rectangular regions to select the rectangular regions passing the uniqueness detection; and performing positioning accuracy detection on the rectangular regions passing the uniqueness detection to select the rectangular regions passing the positioning accuracy detection as the positioning core region extracted from the candidate region. According to the application, the positioning core region can be automatically extracted from the reference image, and the positioning core region has good extraction effect.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of machine vision, in particular to a positioning core region automatic extraction method, device, equipment and medium for image matching. BACKGROUND

[0002] Image matching is an algorithm for automatically finding a specified pattern in an image or a specified part in an object in the technical field of machine vision. The algorithm first marks and frames a positioning core region in a reference image to make a model, and then finds the position and angle of the model in a target image. Image matching is generally divided into "gray-scale matching" and "feature matching". At present, the mainstream of gray-scale matching method is Correlation-Based (correlation-based) gray-scale value matching. This method uses a normalized cross-correlation function value to evaluate the similarity between a template image (an image in the positioning core region) and a target image, which is a simple and effective matching algorithm, but is not suitable for matching in the presence of rotation. The feature matching method extracts features (points, lines, etc.) in the template image and the target image respectively and describes the features with parameters, and then uses the described parameters for matching. Point feature (edge point, corner point, interest point, etc.) matching is widely used, and the feature point model usually has scale and rotation invariance, which can be applied to matching in the presence of scaling and rotation. The use steps of these two matching methods are unified as follows: selecting a reference image, extracting a positioning core region on the reference image, creating a positioning core, finding a template on a target image, and using the finding result data.

[0003] At present, the extraction step of the positioning core region on the reference image needs an operator to select a suitable region according to the features on the image combined with his own experience, which cannot meet the application scenarios of full-automatic image matching. Moreover, the quality of the positioning core region depends on the operator's experience, which is highly subjective. For operators with less experience, they may extract a poor positioning core region with poor extraction effect, which cannot accurately position and leads to low accuracy of image matching. SUMMARY

[0004] Therefore, it is necessary to provide a positioning core region automatic extraction method, device, equipment and medium for image matching, which can automatically extract a positioning core region and has good extraction effect, in order to solve the technical problem of low accuracy of image matching.

[0005] A positioning core region automatic extraction method for image matching, comprising:

[0006] obtaining a set reference image and image matching data;

[0007] determining a candidate region of the reference image according to the image matching data;

[0008] extracting a positioning core region from the alternative region, comprising:

[0009] performing feature extraction on the alternative region to generate a set of feature points;

[0010] determining a plurality of minimum circumscribed positive rectangles according to the set of feature points to obtain a plurality of rectangular regions;

[0011] performing uniqueness detection on the rectangular regions to select a rectangular region passing the uniqueness detection;

[0012] performing positioning accuracy detection on the rectangular region passing the uniqueness detection, and selecting a rectangular region passing the positioning accuracy detection as the positioning core region extracted from the alternative region.

[0013] In one embodiment, the image matching data includes a number of positioning core regions, and the determining of the alternative region of the reference image according to the image matching data comprises:

[0014] performing gridization on the reference image to obtain a plurality of grids;

[0015] selecting a number of grids from all the grids to form a grid combination, and storing a plurality of grid combinations in a combination set;

[0016] traversing the combination set, taking each grid in the current grid combination as an alternative region, and performing the step of extracting a positioning core region from the alternative region;

[0017] After the step of extracting a positioning core region from the alternative region, the method further comprises:

[0018] if all the alternative regions in the current grid combination have extracted positioning core regions, exiting the traversal and outputting the extracted positioning core regions.

[0019] In one embodiment, the performing of the gridization on the reference image to obtain a plurality of grids comprises:

[0020] gridizing the reference image into a plurality of grids with equal number of rows and columns according to the number of positioning core regions; wherein the number of rows or columns satisfies:

[0021]

[0022] wherein n is the number of rows or columns, and num is the number of positioning core regions.

[0023] In one embodiment, the selecting of a number of grids from all the grids to form a grid combination, and the storing of a plurality of grid combinations in a combination set, comprises:

[0024] numbering the grid;

[0025] selecting a number of the positioning core regions to combine, to obtain a plurality of grid combinations;

[0026] respectively calculating the sum of distances between the grids corresponding to the numbers in each grid combination, and screening the grid combination with the largest sum of distances to store in a combination set.

[0027] In one embodiment, the image matching data includes an image matching method; and the feature extraction on the candidate region to generate a set of feature points includes:

[0028] If the image matching method is feature point matching, a stored feature point extractor interface is called to perform feature extraction on the candidate region, and the extracted feature points are stored in the set of feature points;

[0029] If the image matching method is grayscale matching, a window of a preset size is used to slide the candidate region according to a preset step, and the entropy value of the image in each step window is calculated, and the center point coordinates of each step window are recorded;

[0030] determining the maximum entropy value, and screening the entropy values greater than the maximum entropy value by a preset multiple;

[0031] the center point coordinates of the window corresponding to the screened entropy value are stored as feature points in the set of feature points.

[0032] In one embodiment, the determination of a plurality of minimum circumscribed rectangles according to the set of feature points to obtain a plurality of rectangular regions includes:

[0033] using a density-based clustering algorithm to cluster the set of feature points to obtain a plurality of clusters;

[0034] respectively generating a minimum circumscribed rectangle of each cluster to obtain a plurality of rectangular regions.

[0035] In one embodiment, the image matching data includes matching method corresponding parameters; and the uniqueness detection on the rectangular region to select the rectangular region passing the uniqueness detection includes:

[0036] creating a positioning core using the reference image and the current rectangular region;

[0037] using the matching method corresponding parameters and the created positioning core to find the matching positioning core in the target image;

[0038] if the number of matching positioning cores is 1, the current rectangular region passes the uniqueness detection, the current rectangular region is stored in a uniqueness region set, and the created positioning core corresponding to the current rectangular region is stored in a positioning core set.

[0039] taking the next rectangular region as the current rectangular region, and returning to the step of creating the positioning kernel using the reference image and the current rectangular region.

[0040] In one embodiment, the positioning accuracy detection on the rectangular region detected by the uniqueness detection selects the rectangular region detected by the positioning accuracy detection as the positioning kernel region extracted from the candidate region, including:

[0041] obtaining detection images respectively generated after a plurality of magnitudes of geometric transformations on the reference image;

[0042] traversing the set of positioning kernels, respectively finding matched positioning kernels in each detection image using the current positioning kernel and parameters corresponding to the matching method, and taking the region center point coordinates of the matched positioning kernels as positioning coordinates of the corresponding detection image;

[0043] calculating coordinates of the region center point coordinates of the current positioning kernel after the geometric transformations of each detection image, to obtain transformed coordinates of the corresponding detection image;

[0044] calculating the offset between the positioning coordinates and the transformed coordinates corresponding to the same detection image;

[0045] if the offsets corresponding to all detection images are less than or equal to a preset error threshold, selecting the rectangular region corresponding to the current positioning kernel in the set of unique regions as the positioning kernel region extracted from the candidate region, and exiting the traversal of the set of positioning kernels.

[0046] In one embodiment, the obtaining of the detection images respectively generated after a plurality of magnitudes of geometric transformations on the reference image includes:

[0047] adding Gaussian noise to the reference image to obtain a Gaussian noise image;

[0048] combining a plurality of preset translation amounts and a plurality of preset rotation amounts to obtain a plurality of rigid body transformation magnitudes containing translation amounts and rotation amounts;

[0049] respectively performing rigid body transformations on the Gaussian noise image according to each rigid body transformation magnitude to obtain a plurality of detection images.

[0050] An apparatus for automatically extracting a positioning kernel region of image matching, including:

[0051] a data acquisition module configured to acquire a set reference image and image matching data;

[0052] a region determination module configured to determine a candidate region of the reference image according to the image matching data;

[0053] a positioning core extraction module configured to extract a positioning core region from the candidate region;

[0054] a feature extraction unit configured to perform feature extraction on the candidate region to generate a set of feature points;

[0055] a rectangle generation unit configured to determine a plurality of minimum circumscribed positive rectangles according to the set of feature points to obtain a plurality of rectangular regions;

[0056] a uniqueness detection unit configured to perform uniqueness detection on the rectangular regions and select a rectangular region that passes the uniqueness detection;

[0057] a precision detection unit configured to perform positioning precision detection on the rectangular region that passes the uniqueness detection and select a rectangular region that passes the positioning precision detection as the positioning core region extracted from the candidate region.

[0058] A computer device comprising a memory and a processor, the memory storing a computer program, and the processor implementing the following steps when executing the computer program:

[0059] obtaining a set reference image and image matching data;

[0060] determining a candidate region of the reference image according to the image matching data;

[0061] extracting a positioning core region from the candidate region, comprising:

[0062] performing feature extraction on the candidate region to generate a set of feature points;

[0063] determining a plurality of minimum circumscribed positive rectangles according to the set of feature points to obtain a plurality of rectangular regions;

[0064] performing uniqueness detection on the rectangular regions and selecting a rectangular region that passes the uniqueness detection;

[0065] performing positioning precision detection on the rectangular region that passes the uniqueness detection and selecting a rectangular region that passes the positioning precision detection as the positioning core region extracted from the candidate region.

[0066] A computer readable storage medium having a computer program stored thereon, the computer program being executed by a processor to implement the following steps:

[0067] obtaining a set reference image and image matching data;

[0068] determining a candidate region of the reference image according to the image matching data;

[0069] extracting a positioning core region from the candidate region, comprising:

[0070] perform feature extraction on the candidate region to generate a set of feature points;

[0071] determine a plurality of minimum circumscribed positive rectangles according to the set of feature points to obtain a plurality of rectangular regions;

[0072] perform uniqueness detection on the rectangular regions, and select a rectangular region that passes the uniqueness detection;

[0073] perform positioning accuracy detection on the rectangular region that passes the uniqueness detection, and select a rectangular region that passes the positioning accuracy detection as a positioning core region extracted from the candidate region.

[0074] The positioning core region automatic extraction method, device, computer device, and computer readable storage medium for image matching described above, after determining a candidate region of a reference image, perform feature extraction on the candidate region, generate a rectangular region, perform uniqueness detection and positioning accuracy detection on the rectangular region, and thus select a rectangular region that passes the uniqueness detection and positioning accuracy detection as a positioning core region, which realizes automatic extraction of a positioning core region for a reference image and can be applied to an application scenario of full-automatic image matching. Moreover, the extracted positioning core region satisfies uniqueness and positioning accuracy, and compared with an existing manually drawn positioning core region, the extracted positioning core region can avoid extracting a positioning core region that does not satisfy uniqueness and positioning accuracy, has good extraction effect, and thus effectively improves the uniqueness and positioning accuracy of image matching, thereby improving the accuracy of image matching. BRIEF DESCRIPTION OF DRAWINGS

[0075] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or the prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative effort.

[0076] Figure 1 a flowchart of a positioning core region automatic extraction method for image matching in an embodiment;

[0077] Figure 2 a flowchart of extracting a positioning core region in a candidate region in an embodiment;

[0078] Figure 3 a flowchart of determining a candidate region of a reference image according to image matching data in an embodiment;

[0079] Figure 4 a grid division diagram in an embodiment;

[0080] FIG. 5(a) to FIG. 5(d) are schematic diagrams of selected grid numbering in one embodiment;

[0081] Figure 6 FIG. 6 is a flowchart of an automatic extraction method of a positioning nucleus region in image matching in another embodiment;

[0082] Figure 7 FIG. 7 is a flowchart of extracting a positioning nucleus region in an alternative region in another embodiment;

[0083] Figure 8 FIG. 8 is a structural diagram of an automatic extraction device of a positioning nucleus region in image matching in one embodiment;

[0084] Figure 9 FIG. 9 is a structural diagram of a positioning nucleus extraction module in one embodiment. DETAILED DESCRIPTION

[0085] In order to facilitate the understanding of the present application, the present application will be described in more detail below with reference to the relevant drawings. The drawings show embodiments of the present application. However, the present application can be implemented in many different forms, and is not limited to the embodiments described herein. On the contrary, the purpose of providing these embodiments is to make the disclosure of the present application more thorough and comprehensive.

[0086] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs. The terminology used in the specification of the present application is only for the purpose of describing specific embodiments of the present application, and is not intended to limit the present application.

[0087] As used herein, the singular forms "a", "an" and "the" can also include the plural forms, unless the context clearly indicates otherwise. It should also be understood that the terms "comprise / comprising" or "have / having" specify the presence of stated features, integers, steps, operations, components, parts, or combinations thereof, but do not exclude the presence or addition of one or more other features, integers, steps, operations, components, parts, or combinations thereof.

[0088] As described in the background, the operation of extracting the positioning core region of the reference image in the prior art needs the operator to manually select the appropriate region based on experience, which cannot meet the application scenarios of full-automatic image matching, and the operator with less experience may extract a poor positioning core region, resulting in poor extraction effect, such as: 1. The feature information drawn is not rich (a pure color region is drawn), the positioning core region with unstable features (a feature region greatly affected by light is drawn), and the edge region of the image drawn in the image with certain distortion, which will cause the problems of failure to create a model, low robustness of the matching model, poor positioning accuracy, etc. 2. The feature region drawn is not a unique feature region for the entire image, such as multiple feature regions with the same features, which will make the program unable to determine the exact position of matching. Based on this, the present application provides a scheme capable of automatically extracting a positioning core region for image matching and having good extraction effect.

[0089] In one embodiment, as shown in Figure 1 , a positioning core region automatic extraction method for image matching is provided, which comprises:

[0090] S110: Obtain a set of reference images and image matching data.

[0091] The reference image is a reference image required for extracting the positioning core in image matching; the image matching data is a working parameter of image matching, which can include at least one of the number of positioning core regions, a matching method, and a matching method corresponding parameter; wherein the matching method is data used to describe the method used for image matching, the number of positioning core regions refers to the number of positioning core regions to be extracted, and the matching method corresponding parameter includes a matching search score, a search range, etc. Specifically, the reference image and the image matching data can be obtained by user configuration.

[0092] S130: Determine the candidate region of the reference image according to the image matching data.

[0093] The candidate region is a partial region selected in the reference image. The number of candidate regions can be one or multiple.

[0094] S150: Extract the positioning core region in the candidate region.

[0095] Referring to Figure 2 , step S150 comprises steps S151 to S157. For the case of multiple candidate regions, steps S151 to S157 are performed for each candidate region.

[0096] S151: Perform feature extraction on the candidate region to generate a set of feature points.

[0097] Feature extraction on one of the candidate regions can obtain a plurality of feature points, and all the feature points form a feature point set.

[0098] S153: Determine a plurality of minimum circumscribed positive rectangles according to the feature point set, to obtain a plurality of rectangular regions.

[0099] A minimum circumscribed positive rectangle can be determined from a plurality of feature points, and a plurality of minimum circumscribed positive rectangles can be determined based on all feature points in the feature point set. The minimum circumscribed positive rectangle is a minimum circumscribed rectangle parallel to the coordinate axis, which can be determined according to a known minimum circumscribed positive rectangle generation method, for example, by determining the rectangular boundary from the maximum horizontal coordinate, the minimum horizontal coordinate, the maximum vertical coordinate, and the minimum vertical coordinate of each vertex corresponding to the plurality of feature points. The region corresponding to the minimum circumscribed positive rectangle is the rectangular region.

[0100] S155: Perform uniqueness detection on the rectangular region, and select the rectangular region that passes the uniqueness detection.

[0101] The uniqueness detection is to detect whether the rectangular region is a unique feature region in the reference image. For example, the uniqueness of the rectangular region can be detected by detecting whether the positioning kernel created based on the rectangular region can be matched to a unique region in the target image.

[0102] S157: Perform positioning accuracy detection on the rectangular region that passes the uniqueness detection, and select the rectangular region that passes the positioning accuracy detection as the positioning kernel region extracted from the candidate region.

[0103] The positioning accuracy detection is to detect the positioning accuracy of the positioning kernel extracted from the rectangular region. For example, the positioning accuracy can be detected by detecting the deviation between the rectangular region and the region matched by the positioning kernel created based on the rectangular region in the target image, and determining whether the deviation meets the set requirement to determine whether the positioning accuracy meets the requirement.

[0104] Specifically, for a plurality of rectangular regions that pass the uniqueness detection, the positioning accuracy detection is performed on each rectangular region that passes the uniqueness detection in sequence. When the currently detected rectangular region meets the positioning accuracy detection, the rectangular region is selected as the positioning kernel region and the positioning accuracy detection is stopped, so as to ensure that one candidate region extracts one positioning kernel region.

[0105] The positioning core region automatic extraction method of the image matching determines the candidate regions of the reference image, extracts features from the candidate regions, generates rectangular regions, performs uniqueness detection and positioning accuracy detection on the rectangular regions, and selects the rectangular regions that pass the uniqueness detection and positioning accuracy detection as the positioning core regions, so as to automatically extract the positioning core regions of the reference image, which can be applied to an image matching fully automatic application scenario. Moreover, the extracted positioning core regions meet the uniqueness and positioning accuracy, and compared with the existing manual drawing of the positioning core regions, the positioning core regions that do not meet the uniqueness and positioning accuracy can be avoided, the extraction effect of the positioning core regions is good, and thus the uniqueness and positioning accuracy of the image matching are effectively improved, and the image matching accuracy is improved.

[0106] In one of the embodiments, the image matching data includes the number of positioning core regions. Specifically, the number of positioning core regions is at least two. Figure 3 The step S130 includes steps S131 to S135.

[0107] S131: Grid the reference image to obtain a plurality of grids.

[0108] For example, the number of grids can be determined according to the number of positioning core regions, so as to grid the reference image to obtain the grids. Each grid corresponds to a region serving as an extraction unit of the positioning core region.

[0109] S133: Select a number of grids from all the grids to form a grid combination, and store a plurality of grid combinations in a combination set.

[0110] Taking the number of positioning core regions as num for example, a plurality of grid combinations can be formed by selecting num grids from all the grids in various ways.

[0111] S135: Traverse the combination set, and regard the regions where the grids in the current grid combination are located as candidate regions.

[0112] Then, the step S150 is performed. That is, the extraction of the positioning core region is performed on the regions where the grids in the current traversed grid combination are located. Correspondingly, the step S150 further includes a step S160.

[0113] S160: Determine whether all the candidate regions in the current grid combination have extracted the positioning core regions.

[0114] If not, the step S171 is performed, that is, the regions where the grids in the next grid combination are located are regarded as candidate regions, and the step S150 is performed.

[0115] If yes, the step S173 is performed, that is, the traversal is exited, and the extracted positioning core regions are output.

[0116] For example, the extracted positioning core region can be output to a display for display. By traversing the combination set, for the current grid combination, each grid in it is traversed, and the automatic extraction of the positioning core region is performed in the region where the grid is located, until all the regions where the grids in a grid combination can extract the positioning core region, at which time the traversal is exited, and the positioning core region extracted in each grid region in the grid combination is taken as the extraction result.

[0117] In image matching, there are scenarios that require multiple positioning core regions to be extracted. For example, in order to reduce large-resolution image positioning errors and improve matching accuracy, multiple positioning core regions need to be extracted; for example, in the Correlation gray correlation matching method, at least two positioning core regions are needed to create a template and search, and at least two center lines of the rectangular frames found are used to calculate the rotation angle, so when the Correlation gray correlation matching method is used and the rotation angle needs to be output, at least two positioning core regions need to be extracted. In the prior art, when multiple positioning core regions are needed, manual drawing is required, which requires the operator to understand the drawing rules corresponding to the scene, increasing the difficulty of drawing and the complexity of operation. In the present embodiment, an automatic extraction scheme for the number of at least two positioning core regions is provided, which can be applied to scenarios that require multiple positioning core regions to be extracted, and does not require manual drawing, solving the problem of high difficulty and complexity of operation under the demand for multiple positioning core regions.

[0118] In one of the embodiments, step S131 comprises: according to the number of positioning core regions, gridizing the reference image into a plurality of grids with equal number of rows and columns; wherein the number of rows or columns satisfies the following formula 1:

[0119] (Formula 1);

[0120] In the formula, n is the number of rows or columns, and num is the number of positioning core regions; sqrt represents the square root operation; and ceil represents the upward rounding operation. In the present embodiment, the number of rows and columns of gridization is determined according to the number of positioning core regions, and the whole reference image is gridized in the n-row-n-column manner, so that the grid size is appropriate and the division effect is good.

[0121] In one of the embodiments, step S133 comprises steps (a1) to (a3).

[0122] Step (a1): numbering the grids.

[0123] The grid numbering method can be a Z-shaped cumulative numbering method from left to right and from top to bottom starting from 0. For example, the grid numbering of a 3*3 grid is as follows: Figure 4As shown. It is understood that in other embodiments, other numbering methods may also be used, such as numbering each column of the grid from top to bottom in a 1-shape, or numbering each column of the grid sequentially from left to right.

[0124] Step (a2): Select the number of numbered core regions and combine them to obtain multiple grid combinations.

[0125] The C-Combination is performed by selecting numbers based on the number of positioning core regions. For example, when the number of positioning core regions is 3, according to Formula 1, the number of 3*3 grids is 9, meaning that 3 grid numbers are selected from the 9 grid numbers for combination.

[0126] Step (a3): Calculate the sum of distances between the grids corresponding to the numbers in each grid combination, and filter the grid combination with the largest sum of distances to store in the combination set.

[0127] Here, distance refers to the distance between the center points of the corresponding grid cells. The grid combination with the largest sum of distances between cells is selected from all grid combinations; for example, using... Figure 4 Taking the grid shown as an example, the grid combinations with the largest sum of distances are {(0,2,6), (0,2,8), (2,6,8), (0,6,8)}, as shown in Figures 5(a) to 5(d).

[0128] For matching and localization in large-field-of-view images, extracting multiple localization kernel regions aims to effectively reduce the impact of localization errors from individual kernels on the overall image localization result, thereby improving localization accuracy. Research has found that the greater the distance between localization kernel regions, the more significant the effect of reducing localization errors. For example, assuming the use of... Figure 4 The positioning kernel regions extracted from grids 0 and 8 are used for positioning. If the positioning kernel of grid 8 has a large positioning error and positions itself on grid 7, then: the positioning coordinates are the coordinates of the center point of the line connecting grids 0 and 7, and the positioning angle is the angle of the connecting line. If the positioning kernel of grid 8 is replaced with the positioning kernel of grid 4, and the positioning kernel of grid 4 has a positioning error and positions itself on grid 3, then: the positioning coordinates are the coordinates of the center point of the line connecting grids 0 and 3, and the positioning angle is the angle of the connecting line. A comparison shows that the deviation between 0-3 and the actual 0-4 is much larger than the deviation between 0-7 and the actual 0-8. The larger the distance between grids, the smaller the positioning deviation. In this embodiment, by selecting the grid combination with the largest sum of distances and storing it in a combination set, and then extracting the positioning kernel based on the combination set, the positioning deviation of multiple positioning kernels can be reduced, and the positioning accuracy of the positioning kernel can be improved.

[0129] like Figure 6As shown, in one detailed embodiment, a reference image and image matching data configured by a user are acquired, wherein the image matching data includes the number of positioning core regions; then, the reference image is gridded, the grid numbers are defined, and a combination set of grid combinations is generated. Then, each grid combination in the combination set is traversed in turn, for each grid combination, each grid is traversed in turn, and step S150 is performed on each grid as a candidate region, so as to perform automatic extraction of the positioning core region, for the grid region from which the positioning core region is extracted, the result region set is stored, otherwise the result region set is emptied; it is judged whether all grid regions in the currently traversed grid combination are extracted to the positioning core region, if not, the result region set is emptied, the next grid combination is traversed, and automatic extraction of the positioning core region is performed on each grid in the next grid combination, and the cycle is repeated until all grids in a grid combination are extracted to the positioning core region, then the positioning core region corresponding to each grid region in the grid combination stored in the result region set is extracted as the final extracted positioning core region, and the traversal is exited, and the automatic extraction of the multiple positioning core regions is completed.

[0130] In one embodiment, the image matching data includes an image matching method. The image matching method can include feature point matching and grayscale matching. For example, for a visual detection device, the user can select Correlation grayscale correlation matching or Feature Point feature point matching. Specifically, step S151 includes step (b1), or steps (b2) to (b4).

[0131] Step (b1): if the image matching method is feature point matching, a stored feature point extractor interface is called to perform feature extraction on the candidate region, and the extracted feature points are stored in the feature point set.

[0132] The stored feature point extractor can be a feature point extractor provided by the current feature point matching method. In other embodiments, feature extraction can also be performed using an extraction method of the same principle as the current feature point matching method.

[0133] Taking Feature Point feature point matching of a visual detection device as an example, Feature Point is a feature point-based matching provided by Open eVision visual library, and a feature point extractor provided by Open eVision visual library can be used to perform feature extraction on the candidate region.

[0134] Step (b2): if the image matching method is grayscale matching, a window of a preset size is used to slide the candidate region according to a preset step, the entropy value of the image in each step window is calculated, and the center point coordinates of each step window are recorded.

[0135] Step (b3): determining a maximum entropy value, and screening entropy values greater than a preset multiple of the maximum entropy value.

[0136] The preset multiple can be set according to actual needs; specifically, the preset multiple is a value less than 1, for example, the preset multiple can be 0.9. The entropy value of the image can reflect the information amount of the image; from all the entropy values, the entropy values greater than the preset multiple of the maximum entropy value are screened, so that the window image that can reflect more information amount can be selected as much as possible.

[0137] Step (b4): storing the center point coordinates of the window corresponding to the screened entropy value as a feature point into a feature point set.

[0138] Taking the Correlation gray correlation matching of the visual inspection equipment as an example, a window of a preset size is used to step and slide the candidate area, and the entropy value (image entropy) of the image in each step window is calculated, the entropy values of the windows at each step are stored in an entropy value set, and the center point coordinates of the current window are stored in a window point set. After the sliding is completed, the center point coordinates corresponding to the window whose entropy value is greater than 0.9 times the maximum entropy value (the preset multiple is defined according to actual conditions) are counted. The window where the center point coordinates are located has a higher entropy value and can be used as the feature point of Correlation. Finally, the feature point is stored in the feature point set.

[0139] In this embodiment, different feature extraction methods are used for different image matching methods, and the corresponding effective features can be automatically extracted.

[0140] In one of the embodiments, step S153 includes: clustering the feature point set by using a density-based clustering algorithm to obtain a plurality of clusters; and generating a minimum circumscribed rectangle of each cluster to obtain a plurality of rectangular regions.

[0141] The density-based clustering algorithm can include a KANN-DBSCAN clustering method. The KANN-DBSCAN clustering method refers to a method based on K- average nearest neighbor algorithm (KANN, K- Average Nearest Neighbor) and density-based spatial clustering of application with noise (DBSCAN, Density-Based Spatial Clustering of Application with Noise) that adaptively generates a list of density thresholds based on mathematical expectation. It can be understood that in other embodiments, other suitable clustering methods can also be selected according to the clustering target of the feature point set, such as shape-based clustering, size-based clustering, etc.

[0142] In this embodiment, the set of feature points is divided into different feature point subsets based on the idea of density clustering to obtain a cluster, and a rectangular region of a minimum circumscribed positive rectangle corresponding to the cluster is generated, so that the part with dense feature points is extracted and a region is generated, which is a candidate for positioning kernel region, so that a positioning kernel region with rich features can be extracted.

[0143] In one of the embodiments, the image matching data includes parameters corresponding to the matching method. Step S155 includes steps (c1) to (c4).

[0144] Step (c1): creating a positioning kernel using the reference image and the current rectangular region.

[0145] Specifically, the current rectangular region is taken as a positioning kernel region, and a known positioning kernel creation method is used to create a positioning kernel based on the reference image.

[0146] Step (c2): taking the reference image as a target, using the matching method corresponding parameters and the created positioning kernel to find the matching positioning kernel in the target.

[0147] Taking the reference image as a target image, using the matching method corresponding parameters and the created positioning kernel to find the matching positioning kernel in the target image, i.e. the search template operation of image matching.

[0148] Step (c3): if the number of matching positioning kernels is 1, the current rectangular region is detected for uniqueness, the current rectangular region is stored in the uniqueness region set, and the created positioning kernel corresponding to the current rectangular region is stored in the positioning kernel set.

[0149] The number of matching positioning kernels is 1, indicating that the created positioning kernel can match a unique region in the target image, so the corresponding rectangular region satisfies the uniqueness and is detected for uniqueness.

[0150] Step (c4): taking the next rectangular region as the current rectangular region.

[0151] Then, return to step (c1), and thus traverse each rectangular region until the uniqueness detection of all rectangular regions is completed.

[0152] In one of the embodiments, step S157 includes steps (d1) to (d5).

[0153] Step (d1): obtaining detection images respectively generated after performing geometric transformation of multiple amplitudes on the reference image.

[0154] After the reference image is subjected to geometric transformation of one amplitude, a detection image is generated; after the reference image is subjected to geometric transformation of multiple amplitudes, multiple detection images are obtained. The geometric transformation can be rigid transformation, or affine transformation, rotation transformation, etc.

[0155] Step (d2): traversing the positioning core set, using the current positioning core and the corresponding parameters of the matching method to find the matched positioning core in each detection image respectively, and taking the region center point coordinates of the matched positioning core as the positioning coordinates of the corresponding detection image.

[0156] Step (d3): calculating the region center point coordinates of the current positioning core after the geometric transformation of each detection image, to obtain the transformed coordinates of the corresponding detection image.

[0157] For example, assuming that the region center point coordinates of the current positioning core are A, the reference image is subjected to a geometric transformation with a magnitude of B to obtain a first detection image, i.e., the geometric transformation of the first detection image has a magnitude of B, then A is subjected to a geometric transformation with a magnitude of B to obtain coordinates C, and coordinates C are the transformed coordinates corresponding to the first detection image.

[0158] Step (d4): calculating the offset between the positioning coordinates and the transformed coordinates corresponding to the same detection image.

[0159] After steps (d2) and (d3), based on the current positioning core, each detection image corresponds to a positioning coordinate and a transformed coordinate, and the offset between the positioning coordinate and the transformed coordinate of the same detection image is calculated. For example, the offset in the X and Y directions of the positioning coordinate and the transformed coordinate can be calculated, and this offset represents the error of the positioning of the current positioning core.

[0160] Step (d5): if the offsets corresponding to all detection images are less than or equal to the preset error threshold, selecting the rectangular region corresponding to the current positioning core in the uniqueness region set as the positioning core region extracted from the candidate region, and exiting the traversal of the positioning core set.

[0161] The preset error threshold can be set according to actual needs. If the offsets corresponding to all detection images calculated in step (d4) are less than or equal to the preset error threshold, it indicates that the positioning error of the current positioning core is within the allowable range and meets the positioning accuracy requirement, i.e., the rectangular region corresponding to the current positioning core passes the positioning accuracy detection; at this time, the rectangular region corresponding to the current positioning core is taken as the positioning core region extracted from the candidate region, and the traversal of the positioning core set is exited, i.e., the next positioning core is not subjected to positioning accuracy detection, ensuring that one candidate region extracts one positioning core region. Specifically, if there is an offset greater than the preset error threshold among the offsets corresponding to all detection images calculated in step (d4), the next positioning core in the positioning core set is taken as the new current positioning core, and step (d2) is returned to.

[0162] In the embodiment, after the uniqueness detection, the detection image is taken as the target image, and whether the positioning accuracy requirement is met is detected by detecting the deviation between the positioning core created by the detection rectangular region and the positioning core matched in the target image by the created positioning core, thereby providing an automatic positioning accuracy detection method, so that the automatically extracted positioning core region has the indexes of high robustness, high positioning accuracy and meeting the uniqueness.

[0163] In one of the embodiments, the step (d1) comprises: adding Gaussian noise to the reference image to obtain a Gaussian noise image; combining a plurality of preset translation amounts and a plurality of preset rotation amounts to obtain a plurality of rigid body transformation amplitudes containing translation amounts and rotation amounts; and performing rigid body transformation on the Gaussian noise image according to each rigid body transformation amplitude to obtain a plurality of detection images.

[0164] The adding of the Gaussian noise to the reference image can be generating a gray image conforming to the Gaussian distribution, and then adding the gray image to the reference image to obtain the Gaussian noise image. For example, a random gray image conforming to the Gaussian distribution is generated by using the RandGauss function in the intel Ipp library, and the random gray image is added to the original reference image to obtain the Gaussian noise image.

[0165] Taking three translation amounts and three rotation amounts as an example, the translation amount list is [2, 5, 8], and the rotation amount list is [2.0, -3.5, 5.0], wherein the values of the translation amounts and the rotation amounts can be defined according to actual conditions, and the translation amounts are the common translation amounts in the X and Y directions. One value in the translation amount list and one value in the rotation amount list are selected for combination to obtain a set of rigid body transformation amplitudes {[2, 2.0], [2, -3.5], [2, 5.0], [5, 2.0], [5, -3.5], [5, 5.0], [8, 2.0], [8, -3.5], [8, 5.0]}. The set of rigid body transformation amplitudes is traversed, the rigid body transformation matrix of the current amplitude is calculated according to the rotation amount and the translation amount, and then the rigid body transformation is performed on the Gaussian noise image to obtain a detection image.

[0166] In the embodiment, the reference image is first added with Gaussian noise, and then rigid body transformation of different amplitudes is performed, so that a plurality of detection images are obtained from one reference image for positioning accuracy detection, thereby realizing the positioning accuracy detection under the condition of only one reference image.

[0167] For example, Figure 7As shown, in one detailed embodiment, first determine the type of image matching method, if it is the FeaturePoint feature point matching method, then select the feature point extractor provided by the current method or use the feature point extraction method of the same principle to extract the feature points in the current grid to obtain the feature point set; if it is the Correlation gray scale matching method, then define the size of the sliding window and the step size of the step sliding, step sliding the window in the grid, calculate the entropy value of the image in each window, count the maximum entropy value, calculate the entropy threshold value (a preset multiple of the entropy value), and count the center point coordinates of the window with an entropy value greater than the entropy threshold value as the feature points to obtain the feature point set. Then, the feature point set clusters out the clusters, and generates the minimum circumscribed rectangle corresponding to each cluster and stores it in the clustering area set; traverse the clustering area set, and after uniquely determining each minimum circumscribed rectangle, obtain the detection image for positioning accuracy detection, and perform positioning accuracy detection based on the detection image.

[0168] It should be understood that, although Figures 1-3 the steps in the flowchart of the method are shown in sequence according to the arrows, these steps are not necessarily executed in sequence according to the arrows. Unless otherwise specified herein, the execution of these steps is not strictly limited in sequence, and these steps can be executed in other sequences. Moreover, Figures 1-3 at least part of the steps in the method can include multiple steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution sequence of these steps or stages is not necessarily sequential, but can be executed in rotation or alternation with other steps or steps or stages in other steps.

[0169] In one embodiment, as Figure 8 shown, an image matching positioning core area automatic extraction device is provided, including a data acquisition module 810, a region determination module 830, and a positioning core extraction module 850. Wherein:

[0170] The data acquisition module 810 is configured to acquire the set reference image and image matching data; the region determination module 830 is configured to determine the candidate region of the reference image according to the image matching data; and the positioning core extraction module 850 is configured to extract the positioning core region in the candidate region.

[0171] Wherein, referring to Figure 9The positioning core region extraction module 850 includes a feature extraction unit 851, a rectangle generation unit 853, a uniqueness detection unit 855, and a precision detection unit 857. The feature extraction unit 851 is configured to perform feature extraction on the candidate region to generate a set of feature points. The rectangle generation unit 853 is configured to determine a plurality of minimum circumscribed positive rectangles according to the set of feature points to obtain a plurality of rectangular regions. The uniqueness detection unit 855 is configured to perform uniqueness detection on the rectangular regions and select rectangular regions that pass the uniqueness detection. The precision detection unit 857 is configured to perform positioning precision detection on the rectangular regions that pass the uniqueness detection and select rectangular regions that pass the positioning precision detection as the positioning core region extracted from the candidate region.

[0172] The positioning core region automatic extraction apparatus for image matching determines the candidate region of the reference image, performs feature extraction on the candidate region, generates rectangular regions, performs uniqueness detection and positioning precision detection on the rectangular regions, and selects rectangular regions that pass the uniqueness detection and the positioning precision detection as the positioning core region, thereby realizing automatic extraction of the positioning core region from the reference image and being applicable to an application scenario of full-automatic image matching. Moreover, the extracted positioning core region satisfies uniqueness and positioning precision, can avoid extraction of a positioning core region that does not satisfy uniqueness and positioning precision, and has good extraction effect, thereby effectively improving the uniqueness and positioning precision of image matching and improving the accuracy of image matching.

[0173] For specific limitations of the positioning core region automatic extraction apparatus for image matching, refer to the limitations of the positioning core region automatic extraction method for image matching described above, which will not be repeated here. Each module in the positioning core region automatic extraction apparatus for image matching can be realized by software, hardware, or a combination thereof. Each module described above can be embedded in or independent of a processor in a computer device in hardware form, or can be stored in a memory in a computer device in software form, so as to be called and executed by a processor to perform operations corresponding to each module. It should be noted that the division of modules in the embodiments of the present application is illustrative, and is only a logical functional division. Actual implementation can have another division manner.

[0174] In one embodiment, a computer device is provided, including a memory and a processor, the memory storing a computer program, and the processor implementing the steps in each method embodiment described above when executing the computer program.

[0175] The computer device can automatically extract the positioning core region and can be applied to an image matching fully automatic application scenario, and the extracted positioning core region satisfies uniqueness and positioning accuracy, the extraction effect of the positioning core region is good, and thus the uniqueness and positioning accuracy of image matching are effectively improved, and the image matching accuracy is improved.

[0176] In one embodiment, a computer readable storage medium is provided, and the computer readable storage medium stores a computer program. The computer program is executed by a processor to implement the steps in the above-mentioned method embodiments.

[0177] The computer readable storage medium stores the computer program, and the computer program can implement the steps in the above-mentioned method embodiments. Similarly, the computer readable storage medium can automatically extract the positioning core region and can be applied to an image matching fully automatic application scenario. Moreover, the extracted positioning core region satisfies uniqueness and positioning accuracy, the extraction effect of the positioning core region is good, and thus the uniqueness and positioning accuracy of image matching are effectively improved, and the image matching accuracy is improved.

[0178] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer readable storage medium. When the computer program is executed, the computer program can include the processes in the above-mentioned embodiments. Any reference to a memory, storage, database or other medium in the embodiments provided in the present application can include at least one of a non-volatile and volatile memory. The non-volatile memory can include a read-only memory (ROM), a tape, a floppy disk, a flash memory or an optical memory. The volatile memory can include a random access memory (RAM) or an external cache memory. As an illustration but not limitation, the RAM can be in various forms, such as a static random access memory (SRAM) or a dynamic random access memory (DRAM).

[0179] In the description of the present specification, the description of the terms "some embodiments", "other embodiments", "ideal embodiments" and the like means that the specific features, structures, materials or characteristics described in combination with the embodiments or examples are included in at least one embodiment or example of the present application. In the present specification, the illustrative description of the above terms does not necessarily mean the same embodiment or example.

[0180] Any combination of the technical features in the above embodiments can be made, and for the sake of brevity, not all possible combinations are described above, however, as long as the combination of the technical features does not exist in contradiction, it shall be considered within the scope of the present disclosure.

[0181] The above embodiments only express several implementation manners of the present application, and the description is relatively specific and detailed, but it shall not be understood as a limitation on the patent scope of the present application. It shall be pointed out that, for ordinary skilled persons in the art, several modifications and improvements can be made without departing from the concept of the present application, and these shall be within the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the appended claims.

Claims

1. A positioning core region automatic extraction method of image matching, characterized by, The method comprises the following steps: acquiring a set reference image and image matching data; determining candidate regions of the reference image according to the image matching data; extracting a positioning core region in the candidate region, comprising: performing feature extraction on the candidate region to generate a set of feature points; determining a plurality of minimum circumscribed positive rectangles according to the set of feature points to obtain a plurality of rectangular regions; performing uniqueness detection on the rectangular regions to select rectangular regions passing the uniqueness detection; performing positioning accuracy detection on the rectangular regions passing the uniqueness detection to select a rectangular region passing the positioning accuracy detection as the positioning core region extracted from the candidate region; wherein the image matching data comprises matching method corresponding parameters; the step of performing uniqueness detection on the rectangular regions to select rectangular regions passing the uniqueness detection comprises: creating a positioning core using the reference image and the current rectangular region; using the matching method corresponding parameters and the created positioning core to find matching positioning cores in the target using the reference image as the target; if the number of matching positioning cores is 1, the current rectangular region passes the uniqueness detection, the current rectangular region is stored in a uniqueness region set, and the created positioning core corresponding to the current rectangular region is stored in a positioning core set; the next rectangular region is taken as the current rectangular region, and the step of creating a positioning core using the reference image and the current rectangular region is returned.

2. The method of claim 1, wherein, The image matching data comprises the number of positioning core regions, and the step of determining candidate regions of the reference image according to the image matching data comprises: performing gridization on the reference image to obtain a plurality of grids; selecting a number of grids equal to the number of positioning core regions from all grids to form a grid combination, and storing a plurality of grid combinations in a combination set; traversing the combination set, taking each grid in the current grid combination as a candidate region, and performing the step of extracting a positioning core region in the candidate region; after the step of extracting a positioning core region in the candidate region, the method further comprises: if all candidate regions in the current grid combination have extracted positioning core regions, exiting the traversal and outputting the extracted positioning core regions.

3. The method of claim 2, wherein, The step of performing gridization on the reference image to obtain a plurality of grids comprises: gridizing the reference image into a plurality of grids with equal number of rows and columns according to the number of positioning core regions; wherein the number of rows or columns satisfies: ； wherein n is the number of rows or columns, num is the number of positioning core regions, sqrt represents the square root operation, and ceil represents the upward rounding operation.

4. The method of claim 2, wherein, The step of selecting a number of grids equal to the number of positioning core regions from all grids to form a grid combination and storing a plurality of grid combinations in a combination set comprises: numbering the grids; selecting a number of numbering equal to the number of positioning core regions to form a plurality of grid combinations; calculating the sum of distances between the grids corresponding to the numbering in each grid combination, and selecting the grid combination with the largest sum of distances and storing it in the combination set.

5. The method of claim 1, wherein, The image matching data comprises an image matching method; the step of performing feature extraction on the candidate region to generate a set of feature points comprises: If the image matching method is feature point matching, a stored feature point extractor interface is called to perform feature extraction on the candidate region, and the extracted feature points are stored in a feature point set; If the image matching method is grayscale matching, a window of a preset size is used to slide over the candidate region according to a preset step size, the entropy value of the image in each step window is calculated, and the center point coordinates of each step window are recorded; A maximum entropy value is determined, and entropy values greater than the maximum entropy value by a preset multiple are screened out; The center point coordinates of the window corresponding to the screened entropy value are stored in the feature point set as feature points.

6. The method of claim 1, wherein, The plurality of minimum circumscribed rectangles are determined according to the feature point set, and a plurality of rectangular regions are obtained, including: A density-based clustering algorithm is used to cluster the feature point set, and a plurality of clusters are obtained; A minimum circumscribed rectangle of each cluster is generated, and a plurality of rectangular regions are obtained.

7. The method of claim 1, wherein, The rectangular regions that pass the uniqueness detection are subjected to positioning accuracy detection, and the rectangular regions that pass the positioning accuracy detection are selected as the positioning core regions extracted from the candidate region, including: A plurality of detection images generated after a plurality of amplitude geometric transformations of the reference image are obtained; The positioning core set is traversed, and the current positioning core and the corresponding parameters of the matching method are used to find matching positioning cores in each detection image, and the region center point coordinates of the matching positioning cores are used as the positioning coordinates of the corresponding detection image; The region center point coordinates of the current positioning core are calculated after the geometric transformations of each detection image, and the transformation coordinates of the corresponding detection image are obtained; The offset between the positioning coordinates and the transformation coordinates corresponding to the same detection image is calculated; If the offsets corresponding to all detection images are less than or equal to a preset error threshold, the rectangular region corresponding to the current positioning core is selected from the uniqueness region set as the positioning core region extracted from the candidate region, and the traversal of the positioning core set is exited.

8. The method of claim 7, wherein, The plurality of detection images generated after a plurality of amplitude geometric transformations of the reference image are obtained, including: Gaussian noise is added to the reference image to obtain a Gaussian noise image; A plurality of translation amounts and a plurality of rotation amounts are combined to obtain a plurality of rigid transformation amplitudes containing translation amounts and rotation amounts; Rigid transformations are performed on the Gaussian noise image according to each rigid transformation amplitude to obtain a plurality of detection images.

9. An automatic extraction device for the localization kernel region in image matching, characterized in that, including: A data acquisition module is configured to acquire a set reference image and image matching data; The image matching data includes matching method corresponding parameters; A region determination module is configured to determine a candidate region of the reference image according to the image matching data; A positioning core extraction module is configured to extract a positioning core region in the candidate region; The positioning core extraction module includes: A feature extraction unit is configured to perform feature extraction on the candidate region to generate a feature point set; A rectangle generation unit is configured to determine a plurality of minimum circumscribed rectangles according to the feature point set to obtain a plurality of rectangular regions; The uniqueness detection unit is configured to perform uniqueness detection on the rectangular regions, and select rectangular regions passing the uniqueness detection, including: creating a positioning core using the reference image and a current rectangular region; using the matching method corresponding parameters and the created positioning core to find a matching positioning core in the target using the reference image as the target; if the number of the matching positioning cores is 1, the current rectangular region passes the uniqueness detection, the current rectangular region is stored in a uniqueness region set, and the current rectangular region corresponding created positioning core is stored in a positioning core set; the next rectangular region is used as the current rectangular region, and the reference image and the current rectangular region are used to create a positioning core again until the uniqueness detection of all the rectangular regions is completed; The precision detection unit is configured to perform positioning precision detection on the rectangular regions passing the uniqueness detection, and select rectangular regions passing the positioning precision detection as the positioning core regions extracted from the candidate regions. 10.A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer device is configured to perform the method according to any one of claims 1-8 when the computer program is executed by the processor. The computer program is executed by the processor to implement the steps of the method in any one of claims 1 to 8.

11. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method in any one of claims 1 to 8.

Images