Image Matching Method, Device and Computer Equipment Based on Multi-Layer Verification Mechanism

Abstract

The present invention discloses an image matching method, device and computer device based on a multi-layer inspection mechanism. The method includes the following steps: S1, respectively obtaining a source image and a template image; S2, preprocessing the source image and the template image; S3, creating a source image pyramid structure with an angle of 0 in the manner of gradually reducing the resolution of the image, and rotating the top-layer image with an angle of 0 to generate a complete top-layer source image; S4, constructing a pyramid structure of the template image; S5, using the top-layer template image to match the source images of the M layers at the top of the pyramid, and retaining the first N1 best matching results as the top-layer matching results; S6, extracting candidate matching angles according to the matching results of the M layers at the top layer, and generating a source image pyramid structure corresponding to the angles; S7, processing the upper-layer matching results to obtain the positions to be matched of the source images of the next layer, matching and retaining the best matching results with the number reduced by half as the matching results of this layer; this method improves the image matching effect and efficiency.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision detection, and particularly to an image matching method, device, and computer device based on a multi-layer inspection mechanism. Background Art

[0002] In the field of image processing, contour matching technology is widely used in tasks such as defect detection, target recognition, and image segmentation. Existing contour matching methods usually rely on classical template matching techniques, such as calculating the similarity of multiple contours through Hu Moments. These methods perform image comparison based on different features (such as edges, textures, shapes, etc.), but when faced with complex image deformations, rotations, scale changes, or noise, the matching accuracy often fails to meet high requirements. That is, the existing technologies have the following disadvantages:

[0003] 1. Sensitivity to noise and background complexity

[0004] Existing methods are easily affected in complex backgrounds, especially when there is a lot of image noise. Traditional template matching methods usually rely on global features or edge features of images, which makes them particularly sensitive to background noise, illumination changes, or occlusion. Under these conditions, the matching accuracy drops severely, and false matching or missed detection may occur.

[0005] 2. Low matching accuracy and difficulty in dealing with image deformations

[0006] Existing contour matching methods, such as feature point matching, have a significant reduction in matching accuracy when faced with factors such as rotation, scaling, and deformation of images. Especially when the image undergoes a large-angle rotation or scale change, the matching result is prone to deviate from the real target, resulting in detection failure.

[0007] 3. High computational cost and poor real-time performance

[0008] Traditional sliding window matching methods usually need to calculate the similarity of the source image pixel by pixel, with a huge amount of calculation, especially in high-resolution images. This calculation method leads to a long processing time and cannot meet the requirements of efficient real-time detection. Especially in industrial applications and real-time monitoring systems, the processing efficiency becomes a bottleneck.

[0009] 4. Lack of rotation and scale invariance

[0010] Although some existing contour matching methods can handle rotation and scale changes to a certain extent, they often rely on manually set parameters or preprocess the images (such as rotation, scaling), lacking the ability to automatically adapt to these changes in the algorithm. This not only increases the complexity of the system but also affects the generality and robustness of the algorithm.

[0011] Therefore, how to improve the matching speed while ensuring high precision, especially in the case of complex backgrounds and image deformations, has become a problem in the field of contour matching. Summary of the Invention

[0012] The present invention aims to solve at least one of the technical problems in the related art to some extent. To this end, an object of the present invention is to provide an image matching method, device, and computer device based on a multi-layer inspection mechanism.

[0013] To achieve the above object, in a first aspect, an image matching method based on a multi-layer inspection mechanism according to an embodiment of the present invention includes the steps of:

[0014] S1. Obtain a source image and a template image respectively;

[0015] S2. Preprocess the source image and the template image respectively;

[0016] S3. According to the set number of layers M, create a pyramid structure of the source image with an angle of 0 in the manner of gradually reducing the resolution of the image, and rotate the top-layer image with an angle of 0 to generate a complete top-layer source image with multiple angles, where M is an integer greater than 1;

[0017] S4. Construct a pyramid structure of the template image according to the set number of layers M;

[0018] S5. Use the top-layer template image to match the top M layers of the source image pyramid, and retain the first N1 best matching results as the top-layer matching results, where N1 is an integer greater than 0;

[0019] S6. Extract candidate matching angles according to the matching results of the top M layers, and generate a pyramid structure of the source image corresponding to the angles;

[0020] S7. Use the matching results of the previous layer to obtain the positions to be matched of the source image in the next layer of the pyramid, and retain the best matching results with the quantity reduced by half as the matching results of this layer. Similarly, process these matching results to obtain the positions to be matched in the next lower layer.

[0021] Further, according to an embodiment of the present invention, in step S2, the preprocessing the source image and the template image respectively includes the steps of:

[0022] S201. Perform color space conversion on the source image and the template image;

[0023] S202. Remove salt-and-pepper noise in the source image and the template image through median filtering;

[0024] S203. Binarize the image to convert the source image and the template image into black and white.

[0025] S204. Erode and dilate the binarized source image and template image.

[0026] Further, according to an embodiment of the present invention, in step S3, creating a pyramid structure of the source image in the manner of gradually reducing the resolution of the image according to the set number of layers M includes the steps of:

[0027] S301. According to the set number of layers M, first establish a pyramid of the source image with an angle of 0 by downsampling;

[0028] S302. According to the start and end angles and the angle step set by the user, generate source images of all angles at the top layer by rotating the top-layer source image with an angle of 0;

[0029] S303. Extract the feature information of the top-layer source image, where the feature information includes the picture size, rotation angle, etc.

[0030] Further, according to an embodiment of the present invention, in step S4, constructing a pyramid structure of the template image according to the set number of layers M includes the steps of:

[0031] S401. Generate a pyramid of the template image by downsampling according to the set number of layers M;

[0032] S402. Process each layer of the template image and extract the feature information; where the feature information includes the coordinates of the contour, the number of contour points, the gradient information of the contour points, and the size of the template image.

[0033] Further, according to an embodiment of the present invention, in step S7, using the matching result of the previous layer to match the position to be matched of the source image in the next layer of the pyramid includes the steps of:

[0034] S701. Extract the corresponding matching coordinates and matching angles according to the M matching results at the top layer;

[0035] S702. In the matching of the corresponding angle in the next layer, according to the matching result of the previous layer, expand the area after coordinate transformation and remove the duplicate coordinates to obtain the position to be matched in the next layer, and obtain a series of coordinates;

[0036] S703. Match the coordinate positions obtained in step S702: Combine the template image of the corresponding layer to match the position to be matched of the source image of the corresponding angle in the next layer of the pyramid; After the matching, retain the best matching results with the number reduced by half, and process and optimize to obtain the position to be matched in the next lower layer, and perform the matching in the next lower layer, and loop this process until the final matching result of the bottom layer is obtained.

[0037] Further, according to an embodiment of the present invention, in step S5 and / or step S7, matching the source image includes the steps of:

[0038] S501. Average gray-scale comparison: Calculate the average gray-scale values of the sliding image window and the template image. If the gray-scale difference between the two exceeds the threshold parameter, then eliminate it.

[0039] S502. Comparison of the number of contour points: Calculate the number of contour points of the sliding image window and the template image. If the difference in the number exceeds the parameter threshold, then further exclude the mismatched regions.

[0040] S503. Gradient vector comparison: According to the coordinates and gradient information of the contour points in the already extracted template image, use the Sobel operator to extract the gradient information of the corresponding points at the corresponding coordinate positions in the sliding image window, calculate the dot product of their gradient vectors, and perform normalization calculation of the similarity by dividing by the modulus of the gradient vector. Set the matching results with similarity greater than the set value as valid matching results.

[0041] Further, according to an embodiment of the present invention, in step S5, retaining the best matching result as the matching result of this layer includes the steps of:

[0042] S504. Sort all valid matching results in descending order of similarity.

[0043] S505. Retain the top N1 best matching results in the top layer M as the top layer matching results, and process to obtain the candidate angles. Further, according to an embodiment of the present invention, the number of the best matching results retained in the top layer is: where a, b, and c are respectively constants, L is the number of pyramid layers of the current matching, A_start is the starting angle, A_End is the ending angle, and S is the angle step size.

[0044] Further, according to an embodiment of the present invention, if the gray-scale difference between the two exceeds the parameter threshold, the elimination formula is: where where is the average gray-scale of the template image, is the average gray-scale corresponding to the current sliding image window, is the absolute value of the difference between the two divided by the average gray-scale of the template image;

[0045] where where θ is the algorithm threshold, a and b are constants, n is the number of pyramid layers, and MaxLayer is the maximum number of pyramid layers that can be accepted.

[0046] Further, according to an embodiment of the present invention, if the quantity difference exceeds the parameter threshold, the formula for further excluding the mismatched region is: Wherein, is the number of contour points in the template image, is the number of contour points in the sliding image window image;

[0047] Wherein, where θ is the algorithm threshold, a and b are constants, and MaxLayer is the maximum number of layers acceptable for the pyramid.

[0048] Further, according to an embodiment of the present invention, the similarity formula between the template image and the sliding image window is: Wherein, is the similarity between the template image and the sliding image window when calculating to the Nth point;

[0049] Wherein, Wherein, is the distance between the centroid coordinates of the sliding image window contour and the centroid coordinates of the template image contour; and are the gradient vectors of the template image and the sliding image window at the position respectively; N is the number of currently matched contour points, which is the total number of contour points in the template image in the total similarity calculation.

[0050] Further, according to an embodiment of the present invention, during the process of calculating the similarity , the greedy algorithm is used to restrict the operation. When the deviation of the calculated result exceeds the threshold parameter, the calculation process is stopped, and the calculation formula is: Wherein, Wherein, Wherein, MinScore and Greed are parameter constants respectively; is the number of contour points in the template image, NormMinScore is the normalized minimum score, and NormGreed is the normalized greediness. In the second aspect, an embodiment of the present invention further provides an image matching device based on a multi-layer inspection mechanism, including: an image acquisition module, which is used to acquire the source image and the template image respectively;

[0051] An image preprocessing module, which is used to preprocess the source image and the template image respectively;

[0052] A source image pyramid creation module, which is used to create a pyramid structure of the source image with an angle of 0 in the way of gradually reducing the resolution of the image according to the set number of layers M, and rotate the top-layer image with an angle of 0 to generate a complete top-layer source image with multiple angles, where M is an integer greater than 1; a template image pyramid creation module, which is used to construct a pyramid structure of the template image according to the set number of layers M;

[0053] A pyramid top matching module, which is used to match the pyramid top layer M source image by constructing a template image and retain the first N1 best matching results as the top layer matching results, where N1 is an integer greater than 0; a source image rotation angle pyramid creation module, which is used to extract candidate matching angles according to the top layer M matching results and generate a pyramid structure of the source image corresponding to the angles;

[0054] A pyramid loop recursive matching module, which obtains the positions to be matched of the source image in the next layer of the pyramid by processing the matching results of the previous layer and retains half of the best matching results as the matching results of the next layer. Similarly, the matching results are further optimized to obtain the positions to be matched of the next lower layer, and this process is looped until the final matching results of the bottom layer are obtained. Thirdly, according to the computer device of the embodiment of the present invention, it includes a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, it implements the image matching method based on a multi-layer verification mechanism as described above.

[0055] Image matching method, device and computer device based on a multi-layer inspection mechanism according to an embodiment of the present invention. The method includes the following steps: S1, obtaining a source image and a template image respectively; S2, preprocessing the source image and the template image respectively; S3, creating a pyramid structure of the source image with an angle of 0 in a manner of gradually reducing the resolution of the image according to the set number of layers M, and rotating the top-layer image with an angle of 0 to generate a complete top-layer source image; S4, constructing a pyramid structure of the template image according to the set number of layers M; S5, using the top-layer template image to match the source images of the top M layers of the pyramid, and retaining the first N1 best matching results as the top-layer matching results; S6, extracting candidate matching angles according to the top M matching results, and generating a pyramid structure of the source image corresponding to the angles; S7, processing the top-layer matching results to obtain the positions to be matched of the source images of the next layer of the pyramid, and retaining half of the best matches as the matching results of this layer, and repeating this process until the bottom-layer matching results are obtained. Through image hierarchical processing, the method gradually matches from coarse to fine, reduces the calculation for each layer, and improves the accuracy. Since for each layer of pyramid image, the matching results of the previous layer are recursively passed in turn. As the pyramid progresses layer by layer, the accuracy of the matching process becomes higher and higher. At each layer, only a smaller area of the window needs to be matched, reducing the calculation amount and increasing the speed. In addition, a pyramid structure of the non-top-layer source image is generated according to the angle information transmitted from the top layer, avoiding the generation of unnecessary intermediate images and pyramid layers, and reducing memory consumption; and according to the image features and matching requirements, parameters such as the number of pyramid layers and the size of the sliding window are automatically adjusted to improve the matching effect and efficiency.

[0056] Additional aspects and advantages of the present invention will be given in part in the following description, become apparent in part from the following description, or be learned through the practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on the structures shown in these drawings without creative efforts.

[0058] Figure 1 is a flowchart of an image matching method based on a multi-layer inspection mechanism provided by an embodiment of the present invention;

[0059] Figure 2 is a flowchart of a method for preprocessing the source image and the template image respectively provided by an embodiment of the present invention;

[0060] Figure 3It is a flowchart of a method for creating a pyramid structure of a source image provided by an embodiment of the present invention;

[0061] Figure 4 It is a flowchart of a method for constructing a pyramid structure of a template image provided by an embodiment of the present invention;

[0062] Figure 5 It is a flowchart of a method for matching the M-n source image in the next layer of the pyramid provided by an embodiment of the present invention;

[0063] Figure 6 It is a flowchart of a method for matching each source image at the top layer of the pyramid using three verification methods and retaining the matching results provided by an embodiment of the present invention;

[0064] Figure 7 It is a block diagram of an image matching device based on a multi-layer verification mechanism provided by an embodiment of the present invention;

[0065] Figure 8 It is a block diagram of a computer device provided by an embodiment of the present invention;

[0066] Figure 9 It is a flowchart of a method for creating a pyramid structure of a source image provided by an embodiment of the present invention;

[0067] Figure 10 It is a flowchart of a method for matching each source image at the top layer of the pyramid using three verification methods provided by an embodiment of the present invention;

[0068] Figure 11 It is the algorithm threshold of the function curve graph;

[0069] Figure 12 It is a function curve graph of the optimal number of matching results N1 provided by an embodiment of the present invention;

[0070] Figure 13 It is a schematic diagram of a method for recursively processing layer by layer starting from the top layer M provided by an embodiment of the present invention;

[0071] Figure 14 It is a schematic diagram of the movement of a sliding window of the same size as the template image during the matching process provided by an embodiment of the present invention.

[0072] The realization, functional features and advantages of the object of the present invention will be further described with reference to the embodiments and the accompanying drawings Detailed implementation manners

[0073] Embodiments of the present invention will be described in detail below. Examples of the embodiments are shown in the accompanying drawings, where the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention and should not be construed as limiting the present invention.

[0074] On the one hand, referring to Figure 1 , an embodiment of the present invention provides an image matching method based on a multi-layer inspection mechanism, including the steps of:

[0075] S1. Obtain a source image and a template image respectively;

[0076] S2. Preprocess the source image and the template image respectively;

[0077] S3. According to the set number of layers M, create a pyramid structure of the source image with an angle of 0 in a manner of gradually reducing the resolution of the image, where M is an integer greater than 1;

[0078] S4. Construct a pyramid structure of the template image according to the set number of layers M;

[0079] S5. Use the top-layer template image to match the top M layers of the source image of the pyramid, and retain the first N1 best matching results as the top-layer matching results, where N1 is an integer greater than 0;

[0080] S6. Extract candidate matching angles according to the matching results of the top M layers, and generate a pyramid structure of the source image corresponding to the angles;

[0081] S7. Process the matching results of the previous layer to obtain the positions to be matched of the source image of the next layer of the pyramid, and retain half the number of the best matching results as the matching results of this layer.

[0082] Specifically, as shown in Figure 1 , in step S1, a source image and a template image are obtained respectively; among them, the source image may contain a graphic similar to the template image. Therefore, when matching the contours of the images, it is necessary to use a matching algorithm to find the image in the source image that matches the template image; and record its position, contour and other feature information.

[0083] As shown in Figure 1 and Figure 2As shown in [figures], in step S2, the source image and the template image are preprocessed respectively; wherein, the preprocessing of the source image and the template image respectively includes the steps: S201, performing color space conversion on the source image and the template image; in step S201, through channel filtering processing: performing color space conversion on the source image and the template image, so as to reduce unnecessary color information, focus on the key features of the image, and improve the efficiency of subsequent processing. In step S202, the salt-and-pepper noise in the source image and the template image is removed through median filtering, ensuring the smoothness and structural integrity of the image. Median filtering is performed on the image template and the source image respectively to remove isolated noise points in the image and enhance the stability of image matching. In step S203, the image is binarized, and the source image and the template image are converted into black and white; through binarization processing: the image is binarized, and the image is converted into black and white, so that the target in the image is more prominent and convenient for subsequent contour extraction. The optimal threshold for binarization is automatically selected through the Otsu algorithm to improve the adaptability of binarization processing. In step S204, erosion and dilation processing are performed on the binarized source image and template image. Through erosion and dilation processing: the binarized image is eroded to remove small noise points in the image and refine the target boundary. Then dilation processing is performed to enhance the contour of the target, making the target area more prominent and convenient for subsequent contour matching.

[0084] As Figure 1 and Figure 3 shown in [figures], after the preprocessing is implemented, it is necessary to construct the pyramid structures of the source image and the template image. In the process of constructing the pyramid structure of the source image. Through step S3, according to the set number of layers M, in the way of gradually reducing the resolution of the image, a pyramid structure of the source image with an angle of 0 is created. It includes the steps: S301, according to the set number of layers M, through downsampling, first establish a pyramid of the source image with an angle of 0; create a pyramid structure of the source image, where the bottom layer is the original image and the top layer is the smallest size image. Each layer is generated through downsampling, gradually reducing the resolution of the image. First, establish a pyramid of the source image with an angle of 0. As Figure 9 shown in [figures], each layer in the non-top pyramid only has one image with an angle of 0, while the top pyramid contains images of all angles. And in step S302, according to the start and end angles and the angle step size set by the user, rotate the source image with an angle of 0 at the top layer to obtain a complete rotated pyramid at the top layer. Finally, through step S303, extract the feature information of the source image at the top layer, including the rotation angle, the number of contours, the picture size, etc.

[0085] As Figure 1 and Figure 3As shown in [reference], a pyramid structure of the template image is constructed through step S4. In step S4, according to the set number of layers M, a pyramid structure of the template image is constructed. Specifically, it includes the steps: S401. Generate a pyramid of the template image by downsampling according to the set number of layers M; construct a pyramid structure of the template image according to the set number of layers. S402. Process each layer of the template image and extract feature information; the feature information includes the coordinates of the contour, the number of contour points, the gradient information of the contour points, and the size of the template image. These key pieces of information extracted can be used to improve the matching efficiency in the subsequent matching process.

[0086] As Figure 6 and Figure 10 shown, after the pyramid structures of the source image and the template image are created, in steps S5 and S7, the template image is used to match each layer of the source image in the pyramid. In the embodiment of the present invention, a multi-layer inspection mechanism is used for preliminary screening. Between the source image and the template image at the top layer of the pyramid, a sliding window method is adopted for matching. The size of the window is the same as that of the template image. The process of window sliding is as shown in the physical image of Figure 14 . Each time it slides, a three-layer inspection mechanism is performed to avoid excessive calculations for each slide. Only when all three passes are made will the similarity between the sliding window and the template image be calculated and retained. Specifically, it includes the steps: S501. Average gray-scale comparison: Calculate the average gray-scale values of the sliding image window and the template image. If the gray-scale difference between the two exceeds the parameter threshold, it is eliminated; if the difference between the average gray-scale values of the two exceeds the parameter threshold, it indicates that the similarity between the sliding window and the template image is poor and does not meet the matching requirements. Therefore, the sliding image window needs to be eliminated.

[0087] In an embodiment of the present invention, if the gray-scale difference between the two exceeds the threshold parameter, the elimination formula is: where, is the absolute value of the difference between the average gray-scale of the template image and the average gray-scale corresponding to the sliding image window divided by the average gray-scale of the template image, is the algorithm threshold. Among them, The calculation formula of is: where, is the average gray-scale of the template image, is the average gray-scale corresponding to the sliding image window at this time.

[0088] Among them, the average gray-scale of the template image is: Among them, the average gray-scale corresponding to the sliding image window is: where, The calculation formula of is: where a and b are constants, usually set to 0.2 and 0.3, is the number of pyramid layers, is the maximum number of layers acceptable for the pyramid, generally 7. With such a setting, when the matching is performed at the seventh layer, when the matching is performed at the 0th layer, , it ensures that non-linearly fluctuates stably between 0.2 and 0.5. Among them, the function curve of Figure 11 is as shown in

[0089] As Figure 10 shown in

[0090] In an embodiment of the present invention, if the quantity difference exceeds the parameter threshold, the formula for further excluding the mismatched area is: Among them, is the absolute value of the difference between the number of contour points of the template image and the sliding image window divided by the number of template contour points, is the algorithm threshold. Among them, the calculation formula of is the number of contour points in the template image, is the number of contour points in the sliding image window image;

[0091] Among them, where a and b are constants, usually set to 0.2 and 0.3, is the number of pyramid layers, is the maximum number of layers acceptable for the pyramid, generally 7. With such a setting, when the matching is performed at the seventh layer, when the matching is performed at the 0th layer, it ensures that non-linearly fluctuates stably between 0.2 and 0.5. Among them, the function curve of Figure 11 is as shown in

[0092] As Figure 10As shown, after the sliding image window passes the verification by the number of contour points, the verification of the gradient vector is then required. In step S503, gradient vector comparison: According to the coordinates and gradient information of the contour points in the extracted template image, at the corresponding coordinate positions of the sliding image window, the Sobel operator is used to extract the gradient information of the corresponding points, calculate the dot product of their gradient vectors, and perform normalization calculation of similarity by dividing by the modulus of the gradient vector. The matching results with similarity greater than the set value are set as valid matching results. During the calculation process, whenever the algorithm calculates a new pixel point, the current similarity value S_N is compared with a threshold. If the current similarity value is less than this dynamically adjusted threshold, it means that the matching quality of the current pixel point does not meet the requirements, and the algorithm will stop the calculation and set the similarity to 0.

[0093] Among them, in an embodiment of the present invention, the similarity formula between the template image and the sliding image window is: Among them, is the similarity between the template image and the sliding image window when calculating to the Nth point; and are the gradient vectors of the template image and the sliding image window at the position respectively; N is the number of currently matched contour points, which is the total number of contour points in the template image in the total similarity calculation.

[0094] Among them, Among them, is the distance between the centroid coordinates of the sliding image window contour and the centroid coordinates of the template image contour.

[0095] Furthermore, in an embodiment of the present invention, during the calculation of the similarity , the greedy algorithm is used to restrict the operation. When the deviation of the calculation result exceeds the threshold parameter, the calculation process is stopped, and the calculation formula is: Among them, MinScore is a parameter constant, NormMinScore is the normalized minimum score, which is scaled according to the number of templates so that the influence of the minimum score can be kept consistent under different numbers of templates. NormGreed is the normalized greediness, which adjusts the strength of the greedy strategy according to the parameter and . The value of

[0096] controls the bias of the algorithm to select "better" results during the processing. When calculating to the th pixel point, if the above inequality holds, the operation will end and the similarity will be set to 0.

[0097] Among them, is the number of contour points that have participated in the calculation in the similarity calculation at this time, is when calculating to the th point, the similarity between the template and the window at this time.

[0098] Among them, the normalized minimum score The formula for is: Among them, the normalized greediness The formula for is: Among them, MinScore and Greed are parameter constants; it is found through testing that the effect is better when set to around 0.5 and 0.1. is the number of contour points in the template image.

[0099] After passing through multiple inspection mechanisms, only the matching results with a similarity greater than 0.5 are retained, and the matching positions, angles, and similarity values are recorded.

[0100] Refer to Figure 6 , by sliding the image window and the method verified by multiple inspection mechanisms, after only the matching results in the sliding window that match the image template are retained, it is also necessary to select the best matching result from the retained matching structures in a sorted manner. In steps S5 and S7, retaining the best matching result as the matching result of this layer includes steps: S504, sorting all valid matching results from large to small according to similarity; and in step S505, retaining the top N1 best matching results in the top layer M as the top layer matching result to obtain candidate angles. N1 is a threshold determined according to the user's matching settings, such as the number of pyramid layers, angles, etc. Store these matching positions, corresponding rotation angles, and similarities for reference in subsequent pyramid level matching.

[0101] In an embodiment of the present invention, the number of best matching results is: Among them, a, b, and c are constants respectively, L is the number of pyramid layers of the matching at this time, is the starting angle, is the ending angle, and S is the angle step size.

[0102] Such as Figure 12 shown in, Figure 12It is a function curve graph of the number N1 of the best matching results when the number of layers provided by the embodiments of the present invention is 5, 6, 7, and the candidate angles are 10 and 20. In this image matching algorithm, a = 1.5, b = 1.25, and c = 1 are set. a exists as a multiplier to magnify or shrink the final result. b and c are the powers of L and the candidate angle respectively. When b or c is 1, it represents that we hope to establish a linear relationship. When it is greater than 1, it means that we hope to strengthen the influence of the change of the base number on N. Similarly, when it is less than 1, the influence of the change of the base number on the growth of N is weakened. In this image matching algorithm, we take b = 1.25 to hope that the influence of the number of layers on N is greater. When the number of layers increases, the value of N increases a little more. And taking c = 1 makes the candidate angle have a linear influence on N. L is the number of pyramid layers matched at this time, is the starting angle, is the ending angle, and S is the angle step size.

[0103] After completing the source image matching for all angles of the top layer M, select N1 best matching results as the top layer matching results. After that, it is necessary to continue to match the next M - n layer. Before continuing to match the next layer, it is necessary to regenerate the source image pyramid structure for this layer.

[0104] In step S6, extract the candidate matching angles according to the top layer M matching results, and generate the source image pyramid structure corresponding to the angles; extract the most suitable matching angle according to the top layer matching results. For these angles, generate the source image pyramid corresponding to the angles. This method avoids generating pyramid images for all possible angles, saves computing resources, and improves performance. Generate images at different angles by rotating the image to ensure that the matching effects at different angles are optimized.

[0105] Refer to Figure 1 、 Figure 5 and Figure 12, in step S7, the matching results of the upper layer are processed to obtain the positions to be matched in the source image of the lower layer of the pyramid for matching. It includes the steps: S701, extracting the corresponding matching coordinates and matching angles according to the matching results of the top layer M; and in step S702, in the matching of the corresponding angles in the lower layer, according to the matching results of the upper layer, the neighborhood is expanded after coordinate transformation and duplicate coordinates are removed to obtain the positions to be matched in the lower layer; the matching information is transmitted progressively from the top layer to the bottom layer: according to the matching results of the top layer, the coordinates of the first N matching points and their corresponding angles are transmitted to the lower layer pyramid image of the corresponding angle. In the lower layer, the accuracy of the matching points is further optimized by expanding and calculating the neighborhood of the matching coordinates. The validity of the matching is judged by performing the average gray level test and gradient test as described in S5, and maximum suppression is performed on all neighborhood points to be calculated, duplicate points are screened out, neighborhood points with insufficient similarity are removed, and the positions of the remaining matching points are optimized. Finally, half of the number of matching points compared with the upper layer is retained as the output of this layer to the lower layer. S703, matching the positions to be matched in the source image of the corresponding angle of the lower layer of the pyramid with the template image of the corresponding layer; after the matching is completed, half of the number of the best matching results is retained again, and the positions to be matched in the even lower layer are processed and optimized, and the matching in the even lower layer is performed, and this process is cycled until the final matching results of the bottom layer are obtained. By constructing a template image to match the source image in the neighborhood of the lower layer of the pyramid, in the process of layer-by-layer matching of the pyramid structure of the source image, through layer-by-layer recursive processing: for each layer of the pyramid image, the information of the positions to be matched after processing the matching results of the upper layer is received in turn, and similar information is transmitted to the lower layer after the matching of this layer is completed. As the pyramid progresses layer by layer, the accuracy of the matching process becomes higher and higher. In each layer, only a smaller area window needs to be matched, reducing the computational amount and increasing the speed.

[0106] In addition, during the entire matching process of this image matching method, unnecessary intermediate images and pyramid layers are avoided from being generated, reducing memory consumption. Based on the matching results of the top layer, unnecessary matching angles are removed, and only the source image pyramid of the corresponding angle is generated; the calculation focus of high-level images is optimized, such as improving the speed through parallel computing and further enhancing the performance by using hardware acceleration to achieve performance optimization and expansion. According to the image features and matching requirements, parameters such as the number of pyramid layers and the size of the sliding window are automatically adjusted to improve the matching effect and efficiency. Adaptive adjustment is realized.

[0107] In summary, the image matching method based on a multi-layer inspection mechanism provided by the embodiments of the present invention has the following technical effects:

[0108] 1. Improve the matching speed: By performing hierarchical processing on the image and gradually matching from coarse to fine, the computational amount of each layer is reduced. Especially in the low-resolution layer of the image, a large number of unmatched regions can be quickly excluded, making the overall matching process more efficient.

[0109] 2. Enhance the matching accuracy: In the sub - levels of the pyramid, more refined matching can be carried out, making the final result more accurate. The low levels help quickly locate potential matching regions, and the high levels further optimize and improve the accuracy.

[0110] 3. Better multi - scale adaptability: The image matching algorithm based on the multi - layer mechanism can process templates and source images of different scales and adapt to targets of different sizes. By performing matching at different resolutions, it can effectively overcome the problem of size variations of different objects or templates in the image.

[0111] 4. Reduce redundant calculations: At higher - resolution levels, subsequent matching refinement can be based on the matching results of lower - resolution levels, avoiding full - scale slider matching and calculations at each layer, thus improving the efficiency of the overall algorithm.

[0112] 5. Optimize the utilization of memory and computing resources: Since candidate angles are screened based on the top - level matching, it avoids generating and matching image pyramids of non - candidate angles, requiring less memory and computing resources during processing. The image matching algorithm based on the multi - layer mechanism reduces the pressure on memory and computing resources through hierarchical allocation, meeting the requirements for processing large images.

[0113] Reference Figure 4 On the other hand, the present invention also provides an image matching device based on a multi - layer verification mechanism, including: an image acquisition module, an image pre - processing module, a source image pyramid creation module, a template image pyramid creation module, a pyramid top matching module, a source image rotation angle pyramid creation module, and a pyramid loop recursive matching module. The image acquisition module is used to acquire the source image and the template image respectively;

[0114] The image pre - processing module is used to pre - process the source image and the template image respectively;

[0115] The source image pyramid creation module is used to create a source image pyramid structure with an angle of 0 according to the set number of layers M in the manner of gradually reducing the resolution of the image, and rotate the top - layer image with an angle of 0 to generate a complete top - layer source image with multiple angles, where M is an integer greater than 1;

[0116] The template image pyramid creation module is used to construct a pyramid structure of the template image according to the set number of layers M;

[0117] The pyramid top matching module is used to match the M source images at the top layer of the pyramid by constructing a template image, and retain the top N1 best matching results as the top layer matching results, where N1 is an integer greater than 0;

[0118] The source image rotation angle pyramid creation module is used to extract candidate matching angles according to the top layer M matching results, and generate a source image pyramid structure corresponding to the angles;

[0119] The pyramid loop recursive matching module obtains the positions to be matched of the source images at the next layer of the pyramid by processing the matching results of the previous layer, and retains half of the best matching results in number as the matching results of this layer, and further processes to obtain the positions to be matched of the next lower layer, and loops until the final matching results of the bottom layer image are obtained.

[0120] See Figure 8 Moreover, the present invention further provides a computer device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the above image matching method based on a multi-layer verification mechanism is implemented.

[0121] Exemplarily, the computer program can be divided into one or more modules / units. The one or more modules / units are stored in the memory and executed by the processor to complete the present invention. The one or more modules / units can be a series of computer program instruction segments capable of performing specific functions, and these instruction segments are used to describe the execution process of the computer program in the computer device.

[0122] The computer device may include, but is not limited to, a processor and a memory. Those skilled in the art can understand that the figure is only an example of the computer device, and does not constitute a limitation on the computer device. It may include more or fewer components than shown in the figure, or combine some components, or different components. For example, the computer device may further include input / output devices, network access devices, buses, etc.

[0123] The so-called processor may be a Central Processing Unit (CPU), or may also be other general-purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete pre-set hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0124] The memory may be an internal storage unit of the computer device, such as the hard disk or memory of the computer device. The memory may also be an external storage device of the computer device, such as a plug-in hard disk equipped on the computer device, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, etc. Further, the memory may also include both the internal storage unit and the external storage device of the computer device. The memory is used to store the computer program and other programs and data required by the computer device. The memory may also be used to temporarily store data that has been output or is to be output.

[0125] In a fourth aspect, an embodiment of the present invention further provides a computer storage medium, on which a computer program is stored, and when the program is executed by a processor, the image matching method based on a multi-layer verification mechanism as described above is implemented.

[0126] The computer program may be stored in a computer-readable storage medium. When the computer program is executed by a processor, the steps of the above various method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code may be in the form of source code, object code, executable file, or some intermediate form, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a mobile hard disk, a magnetic disk, an optical disc, a computer memory, a Read-Only Memory (ROM), a Random Access Memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.

[0127] It should be noted that the content included in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice within the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electrical carrier signals and telecommunication signals.

[0128] In the above embodiments, the descriptions of the various embodiments have their own emphases. For the parts not detailed or recorded in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0129] The steps in the method of the embodiments of the present invention can be adjusted, combined, and deleted according to actual needs.

[0130] The modules or units in the system of the embodiments of the present invention can be combined, divided, and deleted according to actual needs.

[0131] Those of ordinary skill in the art can realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be implemented by electronic preset hardware, or a combination of computer software and electronic preset hardware. Whether these functions are executed in the form of preset hardware or software depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present invention.

[0132] In the embodiments provided by the present invention, it should be understood that the disclosed device / computer device 600 and method can be implemented in other ways. For example, the device / computer device 600 embodiments described above are merely illustrative. For example, the division of the modules or units is only a logical function division, and there can be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces, and the indirect couplings or communication connections of the devices or units can be in electrical, mechanical, or other forms.

[0133] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. An image matching method based on a multi-layer verification mechanism, characterized in that Including the steps: S1. Obtain the source image and the template image respectively; S2. Preprocess the source image and the template image respectively; S3. According to the set number of layers M, in the way of gradually reducing the resolution of the image, create a source image pyramid structure with an angle of 0, and rotate the top-layer image with an angle of 0 to generate a complete top-layer source image, where M is an integer greater than 1; S4. Construct a pyramid structure of the template image according to the set number of layers M; S5. Use the top-layer template image to match the source images of the top M layers of the pyramid, and retain the first N1 best matching results as the top-layer matching results, where N1 is an integer greater than 0; S6. Extract the candidate matching angles according to the matching results of the top M layers, and generate a source image pyramid structure corresponding to the angles; S7. Process the upper-layer matching results to obtain the positions to be matched of the source images of the next layer, match the source images of the next layer, and retain half of the number of the best matching results as the matching results of this layer; Among them, in step S5 and / or step S7, the matching of the source image includes the steps: S501. Average gray-scale comparison: Calculate the average gray-scale values of the sliding image window and the template image. If the gray-scale difference between the two exceeds the threshold parameter, eliminate it; S502. Comparison of the number of contour points: Calculate the number of contour points of the sliding image window and the template image. If the number difference exceeds the threshold parameter, further exclude the unmatched areas; S503. Gradient vector comparison: According to the coordinates and gradient information of the contour points in the template image that have been extracted, use the Sobel operator to extract the gradient information of the corresponding points at the corresponding coordinate positions of the sliding image window, calculate the dot product of their gradient vectors, and perform normalization calculation of the similarity by dividing by the modulus of the gradient vector. Set the matching results with a similarity greater than the set value as valid matching results; Among them, in step S5, retaining the first N1 best matching results as the top-layer matching results includes the steps: S504. Sort all valid matching results in descending order of similarity; S505. Retain the first N1 best matching results in the top M layers as the top-layer matching results, and process to obtain candidate angles; Among them, the number of the best matching results reserved at the top layer is as follows: Among them, a, b, and c are constants, L is the number of pyramid layers matched at this time, is the starting angle, is the ending angle, S is the angle step size; among them, if the gray level difference between the two exceeds the threshold parameter, the elimination formula is: Among them, Among them, is the average gray level of the template image, is the average gray level corresponding to the sliding image window at this time, is the absolute value of the difference between the two divided by the average gray level of the template image; among them, Among them, θ is the algorithm threshold, a and b are constants, n is the number of pyramid layers, and MaxLayer is the maximum number of pyramid layers that can be accepted.

2. The image matching method based on a multi-layer inspection mechanism according to claim 1, wherein In step S2, the preprocessing the source image and the template image respectively includes the steps: S201. Perform color space conversion on the source image and the template image; S202. Remove the salt-and-pepper noise in the source image and the template image through median filtering; S203. Binarize the image, and convert the source image and the template image into black and white; S204. Perform erosion and dilation processing on the binarized source image and template image.

3. The image matching method based on a multi-layer inspection mechanism according to claim 1, wherein In step S3, the creating a source image pyramid structure with an angle of 0 according to the set number of layers M in the way of gradually reducing the resolution of the image includes the steps: S301. According to the set number of layers M, through downsampling, first establish a source image pyramid with an angle of 0; S302. According to the start and end angles and the angle step length set by the user, generate the source images of all angles of the top layer by rotating the top-layer source image with an angle of 0; S303. Extract the feature information of the top-layer source image, where the feature information includes the picture size, the number of contours, and the rotation angle.

4. The image matching method based on a multi-layer inspection mechanism according to claim 1, characterized in that In step S4, the construction of the pyramid structure of the template image according to the set number of layers M includes the steps of: S401. Generate the template image pyramid by downsampling according to the set number of layers M. S402. Process each layer of the template image and extract the feature information, where the feature information includes the coordinates of the contour, the number of contour points, the gradient information of the contour points, and the size of the template image.

5. The image matching method based on a multi-layer inspection mechanism according to claim 1, wherein In step S7, the process of obtaining the position to be matched of the source image of the next layer by processing the matching result of the upper layer, matching the source image of the next layer, and retaining half of the best matching results with reduced quantity as the matching result of this layer includes the steps of: S701. Extract the corresponding matching coordinates and matching angles according to the top-layer M matching results. S702. In the matching of the corresponding angle of the next layer, according to the matching result of the upper layer, expand the area after coordinate transformation and remove the repeated coordinates to obtain the position to be matched of the next layer. S703. Combine the template image of the corresponding layer to match the position to be matched of the source image of the corresponding angle of the next layer in the pyramid; after the matching is completed, retain half of the best matching results with reduced quantity, process and optimize to obtain the position to be matched of the next lower layer, and perform the matching of the next lower layer, and loop this process until the final matching result of the bottom layer is obtained.

6. The image matching method based on a multi-layer inspection mechanism according to claim 1, characterized in that If the quantity difference exceeds the threshold parameter, the formula for further excluding the mismatched area is: Wherein, is the number of contour points in the template image, is the number of contour points in the sliding image window image; wherein, where θ is the algorithm threshold, a and b are constants, and MaxLayer is the maximum acceptable number of layers of the pyramid.

7. The image matching method based on a multi-layer inspection mechanism according to claim 1, characterized in that The similarity formula between the template image and the sliding image window is as follows: Wherein, is the similarity between the template image and the sliding image window when calculating to the Nth point; wherein, Wherein, d centroid is the distance between the centroid coordinates of the contour of the sliding image window and the centroid coordinates of the contour of the template image; and are the gradient vectors of the template image and the sliding image window at the position respectively; N is the number of currently matched contour points, which is the total number of contour points in the template image in the total similarity calculation.

8. The image matching method based on a multi-layer inspection mechanism according to claim 7, wherein When calculating the similarity During the process, the greedy algorithm is used to restrict the operation. When the deviation of the calculation result exceeds the threshold parameter, the calculation process is stopped. The calculation formula is as follows: Among them, Among them, Among them, MinScore and Greed are parameter constants respectively; is the number of contour points in the template image, NormMinScore is the normalized minimum score, and NormGreed is the normalized greediness.

9. An image matching device based on a multi-layer inspection mechanism, characterized in that, Implement the image matching method based on the multi-layer verification mechanism as described in claim 1.

10. A computer device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the image matching method based on the multi-layer verification mechanism as described in any one of claims 1 to 8.

Images