Image matching method and device based on multilayer inspection mechanism and computer equipment

Abstract

The invention discloses an image matching method and device based on a multilayer inspection mechanism, and computer equipment. The method comprises the following steps: S1, respectively obtaining a source image and a template image; s2, preprocessing the source image and the template image; s3, creating a source image pyramid structure with an angle of 0 according to a mode of gradually reducing the resolution of the image, and rotating the top-layer image with the angle of 0 to generate a complete top-layer source image; s4, constructing a pyramid structure of the template image; s5, matching M layers of source images of the top layer of the pyramid by using the top layer template image, and reserving the first N1 optimal matching results as top layer matching results; s6, extracting candidate matching angles according to the matching result of the M layers of the top layer, and generating a source image pyramid structure of the corresponding angle; s7, processing an upper-layer matching result to obtain a to-be-matched position of a next-layer source image, and matching and reserving an optimal matching result of which the number is reduced by half as a matching result of the layer; the method improves the image matching effect and efficiency.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision detection, and in particular to an image matching method, device and computer equipment based on a multi-layer inspection mechanism. Background Art

[0002] In the field of image processing, contour matching technology is widely used in tasks such as defect detection, target recognition, and image segmentation. Existing contour matching methods usually rely on classic template matching technology, such as calculating the similarity of multiple groups of contours through Hu moments. These methods compare images based on different features (such as edges, textures, shapes, etc.), but when faced with complex image deformation, rotation, scale changes, or noise, the matching accuracy often cannot meet high requirements. That is, the existing technology has the following disadvantages: 1. Sensitivity to noise and background complexity Existing methods are easily affected in complex backgrounds, especially when there is a lot of image noise. Traditional template matching methods usually rely on global features or edge features of the image, which makes them particularly sensitive to background noise, lighting changes or occlusion. Under these conditions, the matching accuracy is severely reduced, and mismatches or missed detections may occur.

[0003] 2. Low matching accuracy, difficult to deal with image deformation Existing contour matching methods, such as feature point matching, often have significantly reduced matching accuracy when faced with factors such as image rotation, scaling, and deformation. In particular, when the image undergoes a large angle rotation or scale change, the matching results are likely to deviate from the true target, resulting in detection failure.

[0004] 3. High computational overhead and poor real-time performance Traditional sliding window matching methods usually require pixel-by-pixel similarity calculations for source images, which is computationally intensive, especially in high-resolution images. This calculation method results in a long processing time and cannot meet the needs of efficient real-time detection, especially in industrial applications and real-time monitoring systems, where processing efficiency becomes a bottleneck.

[0005] 4. Lack of rotation and scale invariance Although some existing contour matching methods can handle rotation and scale changes to a certain extent, they often rely on manually set parameters or preprocessing of images (such as rotation and scaling), and lack the ability to automatically adapt to these changes in the algorithm. This not only increases the complexity of the system, but also affects the versatility and robustness of the algorithm.

[0006] Therefore, how to improve the matching speed while ensuring high accuracy, especially in the case of complex backgrounds and image deformation, has become a problem in the field of contour matching. Summary of the invention

[0007] The present invention aims to solve one of the technical problems in the related art at least to a certain extent. To this end, the purpose of the present invention is to provide an image matching method, device and computer equipment based on a multi-layer verification mechanism.

[0008] To achieve the above object, in a first aspect, an image matching method based on a multi-layer verification mechanism according to an embodiment of the present invention comprises the steps of: S1, obtain the source image and the template image respectively; S2, preprocessing the source image and the template image respectively; S3, according to the set number of layers M, in a manner of gradually reducing the resolution of the image, creating a source image pyramid structure with an angle of 0, and rotating the top layer image with an angle of 0 to generate a complete top layer source image at multiple angles, wherein M is an integer greater than 1; S4, constructing a pyramid structure of the template image according to the set number of layers M; S5, using the top-layer template image to match the M-layer source images at the top layer of the pyramid, and retaining the first N1 best matching results as the top-layer matching results, where N1 is an integer greater than 0; S6, extracting candidate matching angles according to the matching results of the top M layers, and generating a source image pyramid structure of the corresponding angles; S7, using the matching results of the previous layer to obtain the matching positions of the source image of the next layer of the pyramid, and retaining the best matching results with a reduced number as the matching results of this layer, similarly, processing these matching results and obtaining the positions to be matched in the next layer.

[0009] Further, according to an embodiment of the present invention, in step S2, the preprocessing of the source image and the template image respectively comprises the steps of: S201, performing color space conversion on the source image and the template image; S202, removing salt and pepper noise in the source image and the template image by median filtering; S203, binarizing the image, converting the source image and the template image into black and white; S204, performing corrosion and expansion processing on the binarized source image and template image.

[0010] Further, according to an embodiment of the present invention, in step S3, creating a pyramid structure of the source image in a manner of gradually reducing the resolution of the image according to the set number of layers M comprises the following steps: S301, firstly establish a source image pyramid with an angle of 0 by downsampling according to the set number of layers M; S302, generating source images of all angles of the top layer by rotating the top layer source image with an angle of 0 according to the start and end angles and the angle step set by the user; S303: Extract feature information of the top-level source image, where the feature information includes image size, rotation angle, etc.

[0011] Further, according to an embodiment of the present invention, in step S4, constructing the pyramid structure of the template image according to the set number of layers M includes the following steps: S401, generating a template image pyramid by downsampling according to the set number of layers M; S402, processing each layer of template image and extracting feature information; wherein the feature information includes the coordinates of the contour, the number of contour points, the gradient information of the contour points and the size of the template image.

[0012] Further, according to an embodiment of the present invention, in step S7, matching the to-be-matched position of the source image of the next layer of the pyramid using the matching result of the previous layer includes the steps of: S701, extracting corresponding matching coordinates and matching angles according to the top-level M matching results; S702, in the matching of the corresponding angle of the next layer, according to the matching result of the previous layer, the coordinates are converted, the area is expanded and the repeated coordinates are removed to obtain the position to be matched in the next layer, and a series of coordinates are obtained; S703, matching the coordinate positions obtained in step S702: matching the positions to be matched of the corresponding angle source image of the next layer of the pyramid in combination with the template image of the corresponding layer; after the matching is completed, retaining the best matching results with the number reduced by half, and processing and optimizing to obtain the positions to be matched of the next layer, and matching of the next layer is performed, and this process is repeated until the final matching result of the bottom layer is obtained.

[0013] Further, according to an embodiment of the present invention, in step S5 and / or step S7, matching the source image includes the steps of: S501, average grayscale comparison: calculate the average grayscale value of the sliding image window and the template image, and if the grayscale difference between the two exceeds the threshold parameter, then it is eliminated; S502, contour point quantity comparison: calculate the contour point quantity of the sliding image window and the template image, and if the quantity difference exceeds the parameter threshold, further exclude the mismatching area; S503, gradient vector comparison: according to the coordinates and gradient information of the contour points in the extracted template image, the Sobel operator is used to extract the gradient information of the corresponding points at the corresponding coordinate positions of the sliding image window, and their gradient vector dot products are calculated. The similarity is calculated by normalizing by dividing by the gradient vector modulus, and the matching results with similarity greater than the set value are set as valid matching results.

[0014] Further, according to an embodiment of the present invention, in step S5, retaining the best matching result as the matching result of the layer includes the steps of: S504, sorting all valid matching results from largest to smallest according to similarity; S505: retain the first N1 best matching results in the top layer M as the top layer matching results, and process them to obtain candidate angles. Further, according to one embodiment of the present invention, the number of best matching results retained in the top layer is for: Among them, a, b, c are constants, L is the number of pyramid layers matched at this time, A_start is the starting angle, A_End is the ending angle, and S is the angle step.

[0015] Further, according to an embodiment of the present invention, if the grayscale difference between the two exceeds the parameter threshold, the elimination formula is: in, in, is the average grayscale of the template image, This is the average grayscale corresponding to the sliding image window at this time. It is the absolute value of the difference between the two divided by the average grayscale of the template image; in, Among them, θ is the algorithm threshold, a and b are constants, n is the number of pyramid layers, and MaxLayer is the maximum number of layers acceptable to the pyramid.

[0016] Further, according to an embodiment of the present invention, if the quantity difference exceeds the parameter threshold, the formula for further excluding the mismatched regions is: in, is the number of contour points in the template image, is the number of contour points in the sliding image window image; in, Among them, θ is the algorithm threshold, a and b are constants, and MaxLayer is the maximum number of layers acceptable to the pyramid.

[0017] Further, according to an embodiment of the present invention, the similarity formula between the template image and the sliding image window is: in, To calculate the similarity between the template image and the sliding image window at the Nth point; in, in, is the distance between the centroid coordinates of the sliding image window contour and the centroid coordinates of the template image contour; and are the template image and the sliding image window in The gradient vector of the position; N is the number of contour points currently matched, which is the total number of contour points in the template image in the overall similarity calculation.

[0018] Further, according to one embodiment of the present invention, when calculating the similarity In the process, the greedy algorithm is used to constrain the operation. When the deviation of the calculated result exceeds the threshold parameter, the calculation process is stopped. The calculation formula is: in, in, Among them, MinScore and Greed are parameter constants respectively; is the number of contour points in the template image, NormMinScore is the standardized minimum score, and NormGreed is the standardized greediness. In a second aspect, an embodiment of the present invention further provides an image matching device based on a multi-layer verification mechanism, comprising: an image acquisition module, the image acquisition module is used to respectively acquire a source image and a template image; An image preprocessing module, wherein the image preprocessing module is used to preprocess the source image and the template image respectively; A source image pyramid creation module, the source image pyramid creation module is used to create a pyramid structure of a source image with an angle of 0 according to a set number of layers M, in a manner of gradually reducing the resolution of the image, and rotate the top layer image with an angle of 0 to generate a complete top layer source image at multiple angles, wherein M is an integer greater than 1; a template image pyramid creation module, the template image pyramid creation module is used to construct a pyramid structure of a template image according to the set number of layers M; A pyramid top matching module, which is used to match the M source images at the top layer of the pyramid by constructing a template image, and retain the first N1 best matching results as the top layer matching results, where N1 is an integer greater than 0; a source image rotation angle pyramid creation module, which is used to extract candidate matching angles according to the top layer M matching results, and generate a source image pyramid structure of the corresponding angle; A pyramid cyclic recursive matching module, wherein the pyramid cyclic recursive matching module obtains the positions to be matched of the source image of the next layer of the pyramid by processing the matching results of the previous layer, and retains the best matching results reduced by half as the matching results of the next layer. Similarly, the matching results are optimized to obtain the positions to be matched of the next layer, and this process is repeated until the final matching results of the bottom layer are obtained. In a third aspect, a computer device according to an embodiment of the present invention includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the image matching method based on the multi-layer verification mechanism as described above is implemented.

[0019] According to the image matching method, device and computer equipment based on the multi-layer verification mechanism provided by the embodiment of the present invention, the method comprises the steps of obtaining a source image and a template image respectively through steps S1; preprocessing the source image and the template image respectively; creating a pyramid structure of a source image with an angle of 0 according to a set number of layers M and gradually reducing the resolution of the image according to the set number of layers M, and rotating the top image with an angle of 0 to generate a complete top source image according to the set number of layers M; constructing a pyramid structure of a template image according to the set number of layers M; matching the M-layer source images of the top layer of the pyramid using the top template image, and retaining the first N1 best matching results as the top matching results; extracting candidate matching angles according to the M-layer matching results of the top layer, and generating a source image pyramid structure of the corresponding angle; processing the matching results of the top layer to obtain the to-be-matched position of the source image of the next layer of the pyramid, retaining the best matches with the number reduced by half as the matching results of the layer, and repeating this process until the bottom matching results are obtained; the method performs image layering processing, matches step by step from coarse to fine, reduces the calculation of each layer, and improves the accuracy, because for each layer of the pyramid image, the matching results of the previous layer are recursively transmitted in sequence. As the pyramid progresses layer by layer, the accuracy of the matching process becomes higher and higher. At each layer, only a small area window needs to be matched, which reduces the amount of calculation and increases the speed. In addition, the non-top source image pyramid structure is generated based on the angle information transmitted from the top layer, avoiding the generation of unnecessary intermediate images and pyramid layers, reducing memory consumption; and automatically adjusting the number of pyramid layers, sliding window size and other parameters according to image features and matching requirements to improve matching effect and efficiency.

[0020] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying creative work.

[0022] Figure 1 is a flow chart of an image matching method based on a multi-layer verification mechanism provided by an embodiment of the present invention; Figure 2 is a flow chart of a method for preprocessing the source image and the template image respectively provided by an embodiment of the present invention; Figure 3 is a flow chart of a method for creating a pyramid structure of a source image provided by an embodiment of the present invention; Figure 4 is a flow chart of a method for constructing a pyramid structure of a template image provided by an embodiment of the present invention; Figure 5 It is a flow chart of a method for matching an Mn source image of a next layer of a pyramid provided by an embodiment of the present invention; Figure 6 The embodiment of the present invention provides a flow chart of a method for matching source images at the top layer of a pyramid using three verification methods and retaining matching results; Figure 7 is a structural block diagram of an image matching device based on a multi-layer verification mechanism provided by an embodiment of the present invention; Figure 8 is a structural block diagram of a computer device provided by an embodiment of the present invention; Fig. 9 is a flow chart of a method for creating a pyramid structure of a source image provided by an embodiment of the present invention; Fig.10 It is a flow chart of a method for matching source images at the top layer of a pyramid using three verification methods provided by an embodiment of the present invention; Fig.11 is the algorithm threshold provided by the embodiment of the present invention Function curve graph of ; Fig.12 is a function curve diagram of the number N1 of the best matching results provided by an embodiment of the present invention; Fig.13 It is a schematic diagram of a layer-by-layer recursive processing method starting from the top layer M provided by an embodiment of the present invention; Fig.14 It is a physical schematic diagram of the movement of a sliding window having the same size as the template image during the matching process provided by an embodiment of the present invention.

[0023] The purpose, features and advantages of the present invention will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0024] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.

[0025] On the one hand, see Figure 1 The embodiment of the present invention provides an image matching method based on a multi-layer verification mechanism, comprising the steps of: S1, obtain the source image and the template image respectively; S2, preprocessing the source image and the template image respectively; S3, creating a pyramid structure of the source image with an angle of 0 in a manner of gradually reducing the resolution of the image according to the set number of layers M, wherein M is an integer greater than 1; S4, constructing a pyramid structure of the template image according to the set number of layers M; S5, using the top-layer template image to match the M-layer source images at the top layer of the pyramid, and retaining the first N1 best matching results as the top-layer matching results, where N1 is an integer greater than 0; S6, extracting candidate matching angles according to the matching results of the top M layers, and generating a source image pyramid structure of the corresponding angles; S7, processing the matching results of the previous layer to obtain the to-be-matched positions of the source image of the next layer of the pyramid, and retaining the best matching results with the number reduced by half as the matching results of this layer.

[0026] Specifically, if Figure 1 As shown in , in step S1, a source image and a template image are obtained respectively; wherein the source image may contain a figure similar to the template image. Therefore, when matching the contour of the image, a matching algorithm is required to find an image matching the template image in the source image; and record its position, contour and other feature information.

[0027] like Figure 1 and Figure 2 As shown in , in step S2, the source image and the template image are preprocessed respectively; wherein, the preprocessing of the source image and the template image respectively includes the steps of: S201, color space conversion of the source image and the template image; in step S201, through channel filtering processing: color space conversion of the source image and the template image is performed, thereby reducing unnecessary color information, focusing on the key features of the image, and improving the efficiency of subsequent processing. In step S202, salt and pepper noise in the source image and the template image is removed by median filtering to ensure the smoothness and structural integrity of the image. Median filtering is performed on the image template and the source image respectively to remove isolated noise points in the image and enhance the stability of image matching. In step S203, the image is binarized, and the source image and the template image are converted into black and white; through binarization processing: the image is binarized, and the image is converted into black and white, so that the target in the image is more prominent, which is convenient for subsequent contour extraction. The optimal threshold of binarization is automatically selected by the Otsu algorithm to improve the adaptability of binarization processing. In step S204, the binary source image and template image are corroded and expanded. Through the corroded and expanded processing: the binary image is corroded to remove small noise points in the image and refine the target boundary. Then the expansion processing is performed to enhance the contour of the target, making the target area more prominent, which is convenient for subsequent contour matching.

[0028] like Figure 1 and Figure 3 As shown in , after the preprocessing is implemented, it is necessary to construct a pyramid structure of the source image and the template image. In the process of constructing the pyramid structure of the source image. Through step S3, according to the set number of layers M, in a manner of gradually reducing the resolution of the image, a pyramid structure of the source image with an angle of 0 is created. The steps include: S301, according to the set number of layers M, by downsampling, first establish a source image pyramid with an angle of 0; create a pyramid structure of the source image, in which the bottom layer is the original image and the top layer is the minimum size image. Each layer is generated by downsampling to gradually reduce the resolution of the image. First, establish a source image pyramid with an angle of 0. As shown Fig. 9 As shown in , each layer of the non-top pyramid has only one image with an angle of 0, while the top pyramid contains images of all angles. In step S302, according to the start and end angles and angle step set by the user, the source image with an angle of 0 at the top layer is rotated to obtain a complete top rotated pyramid. Finally, in step S303, feature information of the top source image is extracted, including rotation angle, number of contours, image size, etc.

[0029] like Figure 1 and Figure 3 As shown in , the pyramid structure of the template image is constructed by step S4. In step S4, the pyramid structure of the template image is constructed according to the set number of layers M. Wherein, it includes the steps of: S401, generating a template image pyramid by downsampling according to the set number of layers M; constructing the pyramid structure of the template image according to the set number of layers. S402, processing each layer of the template image and extracting feature information; wherein the feature information includes the coordinates of the contour, the number of contour points, the gradient information of the contour points, and the size of the template image. The extracted key information can be used to improve the matching efficiency in the subsequent matching process.

[0030] like Figure 6 and Fig.10 As shown in FIG. 1 , after the pyramid structure of the source image and the template image is created, in step S5 and step S7, the template image is used to match the source images of each layer of the pyramid. In the embodiment of the present invention, a multi-layer inspection mechanism is used for preliminary screening. A sliding window method is used to match the source image at the top layer of the pyramid with the template image. The size of the window is the same as that of the template image. The window sliding process is as follows: Fig.14As shown in the actual image. Each time a slide is made, a three-layer inspection mechanism is performed to avoid too many calculations for each slide. Only when three times are passed will the similarity between the sliding window and the template image be calculated and retained. Among them, the steps include: S501, average grayscale comparison: Calculate the average grayscale value of the sliding image window and the template image. If the grayscale difference between the two exceeds the parameter threshold, it will be eliminated; if the difference between the average grayscale values ​​of the two exceeds the parameter threshold, it indicates that the similarity between the sliding window and the template image is poor and does not meet the matching requirements. Therefore, it is necessary to eliminate the calculated sliding image window.

[0031] In one embodiment of the present invention, if the grayscale difference between the two exceeds the threshold parameter, the elimination formula is: in, The absolute value of the difference between the average grayscale of the template image and the average grayscale corresponding to the sliding image window divided by the average grayscale of the template image. is the algorithm threshold. The calculation formula is: in, is the average grayscale of the template image, This is the average grayscale corresponding to the sliding image window at this time.

[0032] Among them, the average grayscale of the template image is: Among them, the average grayscale corresponding to the sliding image window is: in, The calculation formula is: Where a and b are constants, usually set to 0.2 and 0.3. is the number of pyramid layers, The maximum number of layers that the pyramid can accept is usually 7. With this setting, when matching is performed on the seventh layer, When matching is done at level 0, , guaranteed It fluctuates nonlinearly and smoothly between 0.2 and 0.5. The function curve is as follows Fig.11 as shown in .

[0033] like Fig.10 As shown in , after the sliding image window passes the average gray value verification, the number of contour points needs to be verified. In step S502, the number of contour points is compared: the number of contour points of the sliding image window and the template image is calculated. If the difference in the number of contour points exceeds the parameter threshold, the unmatched area is further excluded. If the difference in the number of contour points of the two exceeds the parameter threshold, it indicates that the similarity between the sliding window and the template image is poor and does not meet the matching requirements. Therefore, the calculated sliding image window needs to be eliminated.

[0034] In one embodiment of the present invention, if the quantity difference exceeds the parameter threshold, the formula for further excluding the mismatched regions is: in, The absolute value of the difference between the number of contour points of the template image and the sliding image window divided by the number of template contour points, is the algorithm threshold. The calculation formula is: is the number of contour points in the template image, is the number of contour points in the sliding image window image; in, Where a and b are constants, usually set to 0.2 and 0.3. is the number of pyramid layers, The maximum number of layers that the pyramid can accept is usually 7. With this setting, when matching is performed on the seventh layer, When matching is done at level 0, Guaranteed It fluctuates nonlinearly and smoothly between 0.2 and 0.5. The function curve is as follows Fig.11 as shown in .

[0035] like Fig.10 As shown in , after the sliding image window passes the contour point number verification, the gradient vector verification is required. In step S503, the gradient vector comparison: according to the coordinates and gradient information of the contour points in the extracted template image, the gradient information of the corresponding points is extracted using the Sobel operator at the corresponding coordinate position of the sliding image window, and their gradient vector dot product is calculated. The similarity is calculated by normalizing by dividing by the gradient vector modulus, and the matching result with a similarity greater than the set value is set as a valid matching result. During the calculation process, every time the algorithm calculates a new pixel point, the current similarity value S_N is compared with a threshold. If the current similarity value is less than the dynamically adjusted threshold, it means that the matching quality of the current pixel point does not meet the requirements, and the algorithm will stop calculating and set the similarity to 0.

[0036] In one embodiment of the present invention, the similarity formula between the template image and the sliding image window is: in, To calculate the similarity between the template image and the sliding image window at the Nth point; and are the template image and the sliding image window in The gradient vector of the position; N is the number of contour points currently matched, which is the total number of contour points in the template image in the overall similarity calculation.

[0037] in, in, is the distance between the centroid coordinates of the sliding image window contour and the centroid coordinates of the template image contour.

[0038] Furthermore, in one embodiment of the present invention, when calculating the similarity In the process, the greedy algorithm is used to constrain the operation. When the deviation of the calculated result exceeds the threshold parameter, the calculation process is stopped. The calculation formula is: Among them, MinScore is a parameter constant, NormMinScore is the minimum score of the standardization, which is based on the number of templates Scaling is performed so that the influence of the minimum score remains consistent with different numbers of templates. NormGreed is the standardized greediness, which is based on Parameters and To adjust the strength of the greedy strategy. The value of controls the algorithm's bias towards choosing "better" results during processing.

[0039] When calculating the When there are pixels, if the above inequality holds, the operation ends and the similarity is set to 0.

[0040] in, This is the number of contour points that have participated in the similarity calculation at this time. To calculate the When there are points, the similarity between the template and the window is calculated.

[0041] Among them, the standardized minimum score The formula is: Among them, the standardized greediness The formula is: Among them, MinScore and Greed are parameter constants. After testing, it is found that the effect is better when they are set to around 0.5 and 0.1. is the number of contour points in the template image.

[0042] After going through a multi-layer verification mechanism, only the matching results with a similarity greater than 0.5 are retained, and the matching position, angle and similarity value are recorded.

[0043] See also Figure 6, by sliding the image window and verifying the method through the multi-layer verification mechanism, after the matching results with the image template in the sliding window are retained, it is also necessary to select the best matching result from the retained matching structure in a sorted manner. In step S5 and step S7, retaining the best matching result as the matching result of this layer includes the steps of: S504, sorting all valid matching results from large to small according to similarity; and in step S505, retaining the first N1 best matching results in the top layer M as the top layer matching results, and processing to obtain candidate angles. N1 is a threshold determined according to the user's matching settings, such as the number of pyramid layers, angles, etc. These matching positions, corresponding rotation angles and similarities are stored to provide a reference for subsequent pyramid level matching.

[0044] In one embodiment of the present invention, the number of best matching results is: Among them, a, b, c are constants, and L is the number of pyramid layers matched at this time. is the starting angle, is the ending angle, and S is the angle step.

[0045] like Fig.12 As shown in Fig.12 It is a function curve diagram of the best matching result number N1 when the number of layers is 5, 6, and 7, and the candidate angles are 10 and 20, provided by an embodiment of the present invention. In this image matching algorithm, a=1.5, b=1.25, and c=1 are set. a exists as a multiplier to enlarge or reduce the final result. b and c are the powers of L and the candidate angle, respectively. When b or c is 1, it means that we hope to establish a linear relationship. When it is greater than 1, it means that we hope to strengthen the impact of the base change on N. Similarly, when it is less than 1, the impact of the base change on the growth of N is weakened. In this image matching algorithm, we take b=1.25 and hope that the number of layers will have a greater impact on N. When the number of layers increases, the value of N increases a little more. Taking c=1 makes the candidate angle have a linear impact on N. L is the number of pyramid layers matched at this time, is the starting angle, is the ending angle, and S is the angle step.

[0046] After completing the source image matching of all angles of the top layer M, the N1 best matching results are selected as the top layer matching results. After that, it is necessary to continue matching the next Mn layer. Before continuing to match the next layer, it is necessary to regenerate the source image pyramid structure for this layer.

[0047] In step S6, candidate matching angles are extracted based on the top-level M matching results, and a source image pyramid structure corresponding to the angle is generated; the most suitable matching angle is extracted based on the top-level matching results. For these angles, a source image pyramid corresponding to the angle is generated. This method avoids generating pyramid images for all possible angles, saves computing resources, and improves performance. Images of different angles are generated by rotating the image to ensure that the matching effects of different angles are optimized.

[0048] See also Figure 1 , Figure 5 and Fig.12 , in step S7, the matching result of the previous layer is processed to obtain the matching position of the source image of the next layer of the pyramid for matching. It includes the steps of: S701, extracting the corresponding matching coordinates and matching angles according to the matching results of the top layer M; and in step S702, in the matching of the corresponding angle of the next layer, according to the matching results of the previous layer, the coordinates are converted and the field is expanded and repeated coordinates are removed to obtain the matching position of the next layer; progressively transferring matching information from the top layer to the bottom layer: according to the matching results of the top layer, the coordinates of the first N matching points and their corresponding angles are transferred to the corresponding angle of the lower layer pyramid image. In the lower layer, the accuracy of the matching points is further optimized by expanding and calculating the neighborhood of the matching coordinates. The validity of the matching is judged by performing the average grayscale test and gradient test as described in S5, and all the neighborhood points to be calculated are maximized and suppressed, duplicate points are screened to eliminate neighborhood points with insufficient similarity, and the positions of the remaining matching points are optimized, and finally the matching points whose number is reduced by half compared with the previous layer are retained as the output of this layer to the next layer. S703, match the to-be-matched position of the corresponding angle source image of the next layer of the pyramid in combination with the template image of the corresponding layer; after the matching is completed, retain the best matching results with the number reduced by half again, and process and optimize to obtain the to-be-matched position of the next layer, and perform the matching of the next layer, and repeat this process until the final matching result of the bottom layer is obtained. By constructing a template image to match the source image in the neighborhood of the next layer of the pyramid, in the process of matching the pyramid structure of the source image layer by layer, recursive processing is performed layer by layer: for each layer of the pyramid image, the to-be-matched position information processed by the matching result of the previous layer is received in turn, and similar information is passed to the next layer after the matching of this layer is completed. As the pyramid progresses layer by layer, the accuracy of the matching process becomes higher and higher. At each layer, only a window of a smaller area needs to be matched, which reduces the amount of calculation and increases the speed.

[0049] In addition, the image matching method avoids generating unnecessary intermediate images and pyramid layers during the entire matching process, thus reducing memory consumption. Through the matching results of the top layer, unnecessary matching angles are removed, and only the source image pyramid of the corresponding angle is generated; the calculation focus of the high-level image is optimized, such as increasing the speed through parallel computing, and further improving the performance through hardware acceleration, so as to achieve performance optimization and expansion. According to the image features and matching requirements, the parameters such as the number of pyramid layers and the size of the sliding window are automatically adjusted to improve the matching effect and efficiency. Adaptive adjustment is achieved.

[0050] In summary, the image matching method based on the multi-layer verification mechanism provided by the embodiment of the present invention has the following technical effects: 1. Improve matching speed: By processing the image in layers, matching is performed step by step from coarse to fine, reducing the amount of calculation for each layer. Especially in the low-resolution layer of the image, a large number of unmatched areas can be quickly excluded, making the overall matching process more efficient.

[0051] 2. Enhanced matching accuracy: In the subdivision levels of the pyramid, matching can be performed more finely, making the final result more accurate. The low levels help quickly locate potential matching areas, and the high levels further optimize and improve accuracy.

[0052] 3. Better multi-scale adaptability: The image matching algorithm based on the multi-layer mechanism can handle templates and source images of different scales and adapt to targets of different sizes. By matching at different resolutions, it can effectively overcome the problem of size changes of different objects or templates in the image.

[0053] 4. Reduce redundant calculations: At higher resolution levels, subsequent matching refinements can be performed based on lower resolution matching results, avoiding complete slider matching and calculations at each level, thereby improving the efficiency of the overall algorithm.

[0054] 5. Optimize the use of memory and computing resources: Since candidate angles are screened based on top-level matching, the image pyramid of non-candidate angles is avoided, and less memory and computing resources are required for processing. The image matching algorithm based on the multi-layer mechanism reduces the pressure on memory and computing resources through hierarchical allocation, and adapts to the needs of processing large images.

[0055] Ginseng Figure 4 On the other hand, the present invention also provides an image matching device based on a multi-layer verification mechanism, comprising: an image acquisition module, an image preprocessing module, a source image pyramid creation module, a template image pyramid creation module, a pyramid top matching module, a source image rotation angle pyramid creation module and a pyramid loop recursive matching module, wherein the image acquisition module is used to respectively acquire a source image and a template image; The image preprocessing module is used to preprocess the source image and the template image respectively; The source image pyramid creation module is used to create a source image pyramid structure with an angle of 0 in a manner of gradually reducing the resolution of the image according to the set number of layers M, and to rotate the top layer image with an angle of 0 to generate a complete top layer source image at multiple angles, wherein M is an integer greater than 1; The template image pyramid creation module is used to construct a pyramid structure of the template image according to the set number of layers M; The pyramid top matching module is used to match the M source images at the top layer of the pyramid by constructing a template image, and retain the first N1 best matching results as the top layer matching results, where N1 is an integer greater than 0; The source image rotation angle pyramid creation module is used to extract candidate matching angles according to the top-level M matching results, and generate a source image pyramid structure of the corresponding angle; The pyramid recursive matching module obtains the to-be-matched position of the source image of the next layer of the pyramid by processing the matching results of the previous layer, and retains the best matching results with the number reduced by half as the matching results of this layer, and further processes to obtain the to-be-matched position of the next layer, and cyclically processes until the final matching result of the bottom layer image is obtained.

[0056] See also Figure 8 On the other hand, the present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned image matching method based on the multi-layer verification mechanism when executing the computer program.

[0057] Exemplarily, the computer program may be divided into one or more modules / units, which are stored in the memory and executed by the processor to implement the present invention. The one or more modules / units may be a series of computer program instruction segments capable of implementing specific functions, which are used to describe the execution process of the computer program in the computer device.

[0058] The computer device may include, but is not limited to, a processor and a memory. Those skilled in the art will appreciate that the figure is only an example of a computer device and does not constitute a limitation on the computer device. The computer device may include more or fewer components than shown in the figure, or may combine certain components, or different components. For example, the computer device may also include an input / output device, a network access device, a bus, etc.

[0059] The processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete preset hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0060] The memory may be an internal storage unit of the computer device, such as a hard disk or memory of the computer device. The memory may also be an external storage device of the computer device, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the computer device. Furthermore, the memory may include both an internal storage unit and an external storage device of the computer device. The memory is used to store the computer program and other programs and data required by the computer device. The memory may also be used to temporarily store data that has been output or is to be output.

[0061] In a fourth aspect, an embodiment of the present invention further provides a computer storage medium on which a computer program is stored, and when the program is executed by a processor, the image matching method based on the multi-layer verification mechanism as described above is implemented.

[0062] The computer program may be stored in a computer-readable storage medium, and when the computer program is executed by a processor, the steps of each of the above method embodiments may be implemented. The computer program includes computer program code, and the computer program code may be in source code form, object code form, executable file or some intermediate form, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc.

[0063] It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practices in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practices, computer-readable media does not include electrical carrier signals and telecommunication signals.

[0064] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0065] The steps in the method of the embodiment of the present invention can be adjusted in order, combined or deleted according to actual needs.

[0066] The modules or units in the system of the embodiment of the present invention may be combined, divided or deleted according to actual needs.

[0067] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic preset hardware, or in a combination of computer software and electronic preset hardware. Whether these functions are performed in preset hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0068] In the embodiments provided by the present invention, it should be understood that the disclosed device / computer equipment 600 and method can be implemented in other ways. For example, the device / computer equipment 600 embodiment described above is only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0069] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. An image matching method based on a multi-layer verification mechanism, characterized in that: Includes steps: S1, obtain the source image and the template image respectively; S2, preprocessing the source image and the template image respectively; S3, according to the set number of layers M, in a manner of gradually reducing the resolution of the image, creating a source image pyramid structure with an angle of 0, and rotating the top image with an angle of 0 to generate a complete top source image, wherein M is an integer greater than 1; S4, constructing a pyramid structure of the template image according to the set number of layers M; S5, using the top-layer template image to match the M-layer source images at the top layer of the pyramid, and retaining the first N1 best matching results as the top-layer matching results, where N1 is an integer greater than 0; S6, extracting candidate matching angles according to the matching results of the top M layers, and generating a source image pyramid structure of the corresponding angles; S7, processing the upper layer matching results to obtain the to-be-matched positions of the next layer source images, matching the next layer source images and retaining the best matching results with the number reduced by half as the matching results of this layer.

2. The image matching method based on the multi-layer verification mechanism according to claim 1 is characterized in that: In step S2, the preprocessing of the source image and the template image respectively comprises the following steps: S201, performing color space conversion on the source image and the template image; S202, removing salt and pepper noise in the source image and the template image by median filtering; S203, binarizing the image, converting the source image and the template image into black and white; S204, performing corrosion and expansion processing on the binarized source image and template image.

3. The image matching method based on the multi-layer verification mechanism according to claim 1 is characterized in that: In step S3, the step of creating a pyramid structure of the source image in a manner of gradually reducing the resolution of the image according to the set number of layers M includes the following steps: S301, firstly establish a source image pyramid with an angle of 0 by downsampling according to the set number of layers M; S302, generating source images of all angles of the top layer by rotating the top layer source image with an angle of 0 according to the start and end angles and the angle step set by the user; S303, extracting feature information of the top-level source image; wherein the feature information includes image size, number of contours, and rotation angle.

4. The image matching method based on the multi-layer verification mechanism according to claim 1 is characterized in that: In step S4, constructing the pyramid structure of the template image according to the set number of layers M includes the following steps: S401, generating a template image pyramid by downsampling according to the set number of layers M; S402, processing each layer of template image and extracting feature information; wherein the feature information includes the coordinates of the contour, the number of contour points, the gradient information of the contour points and the size of the template image.

5. The image matching method based on the multi-layer verification mechanism according to claim 1 is characterized in that: In step S7, the processing of the upper layer matching results to obtain the to-be-matched positions of the next layer source images, matching the next layer source images and retaining the best matching results with the number reduced by half as the matching results of this layer includes the following steps: S701, extracting corresponding matching coordinates and matching angles according to the top-level M matching results; S702, in the matching of the corresponding angle of the next layer, according to the matching result of the previous layer, the coordinates are converted, the area is expanded and the duplicate coordinates are removed to obtain the position to be matched in the next layer; S703, matching the positions to be matched of the source image of the corresponding angle in the next layer of the pyramid in combination with the template image of the corresponding layer; after the matching is completed, retaining the best matching results with the number reduced by half, and processing and optimizing to obtain the positions to be matched in the next layer, and matching in the next layer is performed, and this process is repeated until the final matching result of the bottom layer is obtained.

6. The image matching method based on a multi-layer verification mechanism according to any one of claims 1 to 5, characterized in that: In step S5 and / or step S7, matching the source image comprises the steps of: S501, average grayscale comparison: calculate the average grayscale value of the sliding image window and the template image, and if the grayscale difference between the two exceeds the threshold parameter, then it is eliminated; S502, contour point quantity comparison: calculate the contour point quantity of the sliding image window and the template image, and if the quantity difference exceeds a threshold parameter, further exclude the mismatching area; S503, gradient vector comparison: according to the coordinates and gradient information of the contour points in the extracted template image, the Sobel operator is used to extract the gradient information of the corresponding points at the corresponding coordinate positions of the sliding image window, and their gradient vector dot products are calculated. The similarity is calculated by normalizing by dividing by the gradient vector modulus, and the matching results with similarity greater than the set value are set as valid matching results.

7. The image matching method based on the multi-layer verification mechanism according to claim 6 is characterized in that: In step S5, retaining the first N1 best matching results as the top matching results includes the following steps: S504, sorting all valid matching results from largest to smallest according to similarity; S505 , retain the first N1 best matching results in the top layer M as the top layer matching results, and process them to obtain candidate angles.

8. The image matching method based on the multi-layer verification mechanism according to claim 7 is characterized in that: The number of best matching results retained at the top level for: Among them, a, b, c are constants, L is the number of pyramid layers matched at this time, A_start is the starting angle, A_End is the ending angle, and S is the angle step.

9. The image matching method based on the multi-layer verification mechanism according to claim 7, characterized in that: If the grayscale difference between the two exceeds the threshold parameter, the elimination formula is: in, in, is the average grayscale of the template image, This is the average grayscale corresponding to the sliding image window at this time, It is the absolute value of the difference between the two divided by the average grayscale of the template image; in, Among them, θ is the algorithm threshold, a and b are constants, n is the number of pyramid layers, and MaxLayer is the maximum number of layers acceptable to the pyramid.

10. The image matching method based on the multi-layer verification mechanism according to claim 7, characterized in that: If the quantity difference exceeds the threshold parameter, the formula for further excluding unmatched areas is: in, is the number of contour points in the template image, is the number of contour points in the sliding image window image; where, Among them, θ is the algorithm threshold, a and b are constants, and MaxLayer is the maximum number of layers acceptable to the pyramid.

11. The image matching method based on the multi-layer verification mechanism according to claim 7, characterized in that: The similarity formula between the template image and the sliding image window is: in, To calculate the similarity between the template image and the sliding image window at the Nth point; in, in, is the distance between the centroid coordinates of the sliding image window contour and the centroid coordinates of the template image contour; and are the template image and the sliding image window in The gradient vector of the position; N is the number of contour points currently matched, which is the total number of contour points in the template image in the overall similarity calculation.

12. The image matching method based on the multi-layer verification mechanism according to claim 11, characterized in that: In calculating similarity In the process, the greedy algorithm is used to constrain the operation. When the deviation of the calculated result exceeds the threshold parameter, the calculation process is stopped. The calculation formula is: in, in, Among them, MinScore and Greed are parameter constants respectively; is the number of contour points in the template image, NormMinScore is the standardized minimum score, and NormGreed is the standardized greediness.

13. An image matching device based on a multi-layer verification mechanism, characterized in that: include: An image acquisition module, wherein the image acquisition module is used to respectively acquire a source image and a template image; An image preprocessing module, wherein the image preprocessing module is used to preprocess the source image and the template image respectively; A source image pyramid creation module, the source image pyramid creation module is used to create a pyramid structure of a source image with an angle of 0 according to a set number of layers M, in a manner of gradually reducing the resolution of the image, and to rotate a top layer image with an angle of 0 to generate a complete top layer source image at multiple angles, wherein M is an integer greater than 1; A template image pyramid creation module, wherein the template image pyramid creation module is used to construct a pyramid structure of the template image according to a set number of layers M; A pyramid top matching module, wherein the pyramid top matching module is used to match the M-layer source images of the top layer of the pyramid using the top layer template image, and retain the first N1 best matching results as the top layer matching results, wherein N1 is an integer greater than 0; A source image rotation angle pyramid creation module, the source image rotation angle pyramid creation module is used to extract candidate matching angles according to the matching results of the top M layers, and generate a source image pyramid structure of the corresponding angle; A pyramid cyclic recursive matching module is used to use the matching results of the previous layer to obtain the to-be-matched position of the source image of the next layer. After the matching is completed, the best matching result with the number reduced by half is retained as the matching result of this layer. Similarly, the matching result is optimized to obtain the to-be-matched position of the next layer. This process is repeated until the final matching result of the bottom layer is obtained.

14. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the image matching method based on the multi-layer verification mechanism as described in any one of claims 1 to 12 is implemented.

Images