Template matching method and device for visual features, equipment and storage medium

Abstract

The invention discloses a template matching method and device for visual features, equipment and a storage medium, and relates to the technical field of template matching, and the method comprises the steps: carrying out the training of a deep classification neural network model through a public image data set, and improving a neural network structure, and building a target template matching model; extracting template image features and to-be-matched image features from a target template image data set and a to-be-matched image through the target template matching model; simulating a sliding window, taking the template image feature as a convolution kernel, and calculating an approximate cosine similarity between the template image feature and the to-be-matched image feature; and determining a template matching result from the approximate cosine similarity according to a maximum pooling operation and a similarity threshold method. According to the method, feature extraction and matching are carried out by using the deep classification neural network model, the matching precision and the calculation efficiency are improved, the generalization ability is enhanced, and the matching error rate is reduced.

Description

Technical Field

[0001] The present application relates to the technical field of template matching, and more specifically, to a visual feature template matching method, device, equipment and storage medium. Background Art

[0002] Template matching technology, as one of the cornerstones of image processing and computer vision, has a wide range of applications, covering multiple key areas from target detection to face recognition, motion tracking and remote sensing image analysis. The core of this technology is that it can accurately locate the area that is most similar to the preset template in complex image data, thereby accurately capturing a specific target. In these diverse application scenarios, template matching technology has demonstrated its irreplaceable and important value.

[0003] The NCC (Normalized Cross-Correlation) algorithm is a method for measuring the similarity between two sequences and is widely used in image processing, pattern recognition, and signal processing. Images are collected by a calibrated binocular camera, or two monocular cameras are combined into a binocular camera. The purpose of the correction is to make the epipolar lines of the two frames of images in the horizontal direction to facilitate subsequent NCC operations. This includes removing image distortion and correcting the position of the optical center. Using the NCC algorithm for matching, the best match is the one with the highest correlation on the same horizontal line with the pixel to be tested in the left view in the right view. Record its disparity d, that is, the difference between the horizontal direction of the pixel to be tested and the horizontal direction of the matching pixel. Using the disparity map obtained from the matching result, the depth map with the left view as the reference system is inverted using similar triangles.

[0004] However, although traditional NCC algorithms have shown excellent stability and reliability in many situations, their performance is significantly limited when faced with complex challenges such as image deformation and occlusion. During the processing of such algorithms, a large number of pixel-level grayscale comparisons and correlation analyses are often required, which not only increases the complexity of the calculation, but also reduces the processing speed, making it difficult for them to meet the stringent requirements of real-time and efficiency in industrial applications. When achieving rotational invariance, it is usually necessary to complete it through multiple matches, which not only increases the computational burden, but also fails to meet the real-time requirements. In practical applications, traditional algorithms can often only calculate one angle, which limits the breadth and depth of their application. Summary of the invention

[0005] In response to at least one defect or improvement need in the prior art, the present invention provides a template matching method, device, equipment and storage medium for visual features, which are used to solve the problem that traditional algorithms in the prior art need to perform a large number of pixel-level grayscale comparisons and correlation analyses when encountering image deformation and occlusion, which not only increases the complexity of calculation, but also reduces the processing speed. In practical applications, only one angle can be calculated, thereby limiting the breadth and depth of its application.

[0006] To achieve the above object, according to a first aspect of the present invention, a template matching method of visual features is provided, comprising:

[0007] Use public image datasets to train deep classification neural network models, and improve the neural network structure to build a target template matching model;

[0008] Extracting template image features and image features to be matched from the target template image dataset and the image to be matched through a target template matching model;

[0009] The simulated sliding window uses the template image features as the convolution kernel to calculate the approximate cosine similarity between the template image features and the image features to be matched;

[0010] The template matching result is determined from the approximate cosine similarity based on the maximum pooling operation and the similarity threshold method.

[0011] In a possible implementation, extracting template image features and to-be-matched image features from a target template image dataset and an image to be matched by a target template matching model includes:

[0012] Establishing an initial template image data set, and preprocessing the initial template image data set to obtain a target template image data set;

[0013] Inputting the target template image data set into the target template matching model to extract multiple template image features;

[0014] The image to be matched is input into the target template matching model to extract the features of the image to be matched.

[0015] In a possible implementation, an initial template image dataset is established, and the initial template image dataset is preprocessed to obtain a target template image dataset, including:

[0016] Cropping multiple basic template images to obtain an initial template image dataset of a preset shape and a preset size;

[0017] The initial template image data set is rotated to obtain a multi-angle template image data set;

[0018] A target template image dataset is obtained by performing various affine transformations on the multi-angle template image dataset.

[0019] In a possible implementation, the simulated sliding window uses the template image feature as a convolution kernel to calculate the approximate cosine similarity between the template image feature and the image feature to be matched, including:

[0020] The template image features are used as the convolution kernel and the features of the image to be matched are used as the local vector to establish the initial approximate cosine similarity calculation formula;

[0021] The initial approximate cosine similarity calculation formula is approximated according to the template image features and the image features to be matched to obtain the target approximate cosine similarity calculation formula;

[0022] Based on the target approximate cosine similarity calculation formula, the approximate cosine similarity between the image feature to be matched and each template image feature is calculated respectively.

[0023] In a possible implementation, the initial approximate cosine similarity calculation formula is approximated according to the template image features and the image features to be matched to obtain the target approximate cosine similarity calculation formula, including:

[0024] The binary norm of the image feature to be matched in the initial approximate cosine similarity calculation formula is approximated to the binary norm of the template image feature, and a transitional approximate cosine similarity calculation formula is obtained;

[0025] The inner product of the unit vector of the template image feature and the feature of the image to be matched is used as the target approximate cosine similarity calculation formula.

[0026] In a possible implementation, determining a template matching result from an approximate cosine similarity according to a maximum pooling operation and a similarity threshold method includes:

[0027] Generate random numbers and stack them to approximate cosine similarity;

[0028] Extract the local maximum from the superimposed approximate cosine similarity through the maximum pooling operation;

[0029] The target template image matching the image to be matched is determined based on the similarity threshold method and the local maximum.

[0030] In a possible implementation, determining a target template image that matches the image to be matched based on a similarity threshold method and a local maximum value includes:

[0031] Set similarity matching threshold based on historical template matching data;

[0032] Traverse all local maxima and determine the local maximum that exceeds the similarity matching threshold;

[0033] The template image corresponding to the maximum value among the local maxima exceeding the similarity matching threshold is used as the target template image.

[0034] According to a second aspect of the present invention, there is also provided a template matching device for visual features, comprising:

[0035] A model building module, which is configured to train a deep classification neural network model using a public image dataset and improve the neural network structure to build a target template matching model;

[0036] A feature extraction module, which is configured to extract template image features and to-be-matched image features from a target template image dataset and an image to be matched through a target template matching model;

[0037] A similarity calculation module is configured to simulate a sliding window and use the template image feature as a convolution kernel to calculate the approximate cosine similarity between the template image feature and the image feature to be matched;

[0038] The template matching module is configured to determine the template matching result from the approximate cosine similarity according to the maximum pooling operation and the similarity threshold method.

[0039] According to the third aspect of the present invention, there is also provided a template matching device for visual features, which includes at least one processing unit and at least one storage unit, wherein the storage unit stores a computer program, and when the computer program is executed by the processing unit, the processing unit executes the steps of any one of the above-mentioned template matching methods for visual features.

[0040] According to a fourth aspect of the present invention, there is also provided a storage medium storing a computer program executable by an access authentication device, wherein when the computer program runs on the access authentication device, the access authentication device executes any of the steps of the template matching method of visual features described above.

[0041] In general, the above technical solutions conceived by the present invention can achieve the following beneficial effects compared with the prior art:

[0042] The present invention provides a visual feature template matching method. By using a public image data set to train a deep classification neural network model, the neural network structure is improved to establish a target template matching model. The method can capture subtle differences and key information in an image, thereby improving the accuracy of template matching. By extracting features from a target template image data set and an image to be matched, the method can process a variety of image inputs, enhance the generalization ability in practical applications, learn more generalized features in an image, and use template image features as convolution kernels to calculate the approximate cosine similarity of template image features and image features to be matched, thereby significantly improving the calculation efficiency of matching. By setting a similarity threshold, the method can automatically screen out candidate images with a high similarity to template image features during the matching process. The maximum pooling operation can retain the maximum similarity value in a local area, which helps to suppress the influence of noise and interference information on the matching result, thereby reducing the matching error rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0044] Figure 1 A schematic diagram of a flow chart of an embodiment of a template matching method of visual features provided by the present invention;

[0045] Figure 2 A schematic diagram of the structure of an embodiment of a Resnet network provided by the present invention;

[0046] Figure 3 The present invention provides Figure 1 A schematic diagram of a flow chart of an embodiment of step S102;

[0047] Figure 4 The present invention provides Figure 3 A schematic flow chart of an embodiment of step S301;

[0048] Figure 5 The present invention provides Figure 1 A schematic flow chart of an embodiment of step S103;

[0049] Figure 6 The present invention provides Figure 1 A schematic flow chart of an embodiment of step S104;

[0050] Figure 7 The present invention provides Figure 6 A flowchart of an embodiment of step S603;

[0051] Figure 8 A schematic structural diagram of an embodiment of a visual feature template matching device provided by the present invention;

[0052] Fig. 9 A schematic diagram of the structure of a visual feature template matching device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0053] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0054] The terms "first", "second", "third", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices.

[0055] The present invention provides a visual feature template matching method, device, equipment and storage medium, which are described below respectively.

[0056] See also Figure 1 , Figure 1 A flow chart of an embodiment of a template matching method of visual features provided by the present invention is provided. In a specific embodiment of the present invention, a template matching method of visual features is disclosed, comprising:

[0057] S101. Use public image datasets to train a deep classification neural network model, and improve the neural network structure to build a target template matching model;

[0058] S102, extracting template image features and to-be-matched image features from the target template image dataset and the to-be-matched image through a target template matching model;

[0059] S103, simulating a sliding window to use the template image feature as a convolution kernel to calculate the approximate cosine similarity between the template image feature and the image feature to be matched;

[0060] S104 , determining a template matching result from the approximate cosine similarity according to a maximum pooling operation and a similarity threshold method.

[0061] In the above embodiment, a deep classification neural network model is first trained using a large-scale public image dataset to ensure that the model can learn rich and discriminative features in the image. These features can not only contain low-level information such as the basic shape and color of the image, but also contain more advanced semantic information, such as the category of the object, the attributes of the scene, etc. Through training, the model can gradually optimize its weight parameters, thereby extracting image features more accurately.

[0062] After the training is completed, the neural network structure is improved to build a model specifically for target template matching. Model improvements may include but are not limited to adjusting the number of network layers, changing the size of the convolution kernel, introducing new activation functions, etc., to meet the needs of the neural network model to better adapt to the template matching task, thereby obtaining a highly customized target template matching model that can more effectively extract and compare image features.

[0063] Next, the target template matching model is used to extract image features from the target template image dataset and the image to be matched. The image features represent the key information in the image in the form of a high-dimensional vector. Since the target template matching model has been fully trained, it can accurately capture the subtle differences and key features in the image, providing strong support for the subsequent matching process.

[0064] After extracting the features, the template image features are used as the convolution kernel to calculate the approximate cosine similarity between the template image features and the image features to be matched. Approximate cosine similarity is a commonly used measurement method that can measure the angle between two vectors, thereby reflecting the degree of similarity between them. By calculating the approximate cosine similarity, a similarity matrix can be obtained, which contains the similarity information between each position in the template image and the image to be matched.

[0065] Finally, the template matching result is determined from the approximate cosine similarity according to the maximum pooling operation and the similarity threshold method. The maximum pooling operation can retain the maximum similarity value in the local area, thereby suppressing the influence of noise and interference information on the matching result. The similarity threshold method sets a threshold to filter the matching results. Only when the similarity exceeds the threshold, the match is considered successful, thus ensuring the accuracy and reliability of the matching results.

[0066] In a specific embodiment of the present invention, three classic classification networks: ResNet, EfficientNet and RegNet are compared in depth. Based on the similarities and differences of these networks, the effects of different operations in the network modules on the matching effect are analyzed in detail, and how to optimize the network is explored. When selecting the feature network layer, the following principles are followed: first, the spatial semantic information needs to be retained, so the part after the average pooling layer in the classification network is excluded; second, the two-dimensional spatial scale should not be reduced too much compared to the original image to ensure sufficient resolution; finally, the feature network layer should avoid fusing the channel attention module with the local feature smoothing operation. When the image is reduced by 32 times, the matching is prone to errors due to the low resolution. When the image is reduced by 16 times, the matching performance is acceptable, but the performance of each network is different. Specifically, ResNet has a good matching effect when it is reduced by 16 times, and can successfully match all 4 similar targets. However, EfficientNet and RegNet failed to successfully match when selecting the last network layer with a resolution reduced by 16 times. Further comparison of the three networks revealed that ResNet does not contain attention modules and local average pooling operations, while EfficientNet does, and RegNet contains both operations. Therefore, after removing the attention modules and local average pooling modules that are integrated with them, the performance of EfficientNet has been significantly improved. Although RegNet has been improved, there are still matching errors. Therefore, compared with the RegNet network structure, ResNet and EfficientNet networks are more suitable for transformation into feature networks.

[0067] See also Figure 2 , Figure 2 This is a schematic diagram of the structure of an embodiment of the Resnet network provided by the present invention. When constructing a feature network, Resnet is preferred, and a 1 / 16-fold network layer is used as the basis for feature extraction. Figure 2As shown, the multiple layers in the conv4_x area meet the screening criteria. In view of the fact that in deep neural networks, the deeper the number of layers, the richer and more detailed features can be extracted. Therefore, in the present invention, the last layer (including activation operation) in conv4_x is preferred in order to obtain the optimal feature expression. The various variants of Resnet are adjusted accordingly: resnet-18 retains the first 13 layers to reconstruct an efficient feature extraction network; resnet-34 retains the first 27 layers; resnet-50 selects the first 40 layers; resnet-101 retains the first 91 layers; and resnet-152 retains the first 142 layers to reconstruct the feature extraction network. The selection of the two networks, efficientnet and regnet, not only needs to consider the multiples of the network layer output scale, but more importantly, it is also necessary to prudently delete the attention module of the corresponding layer. In order to illustrate this point more intuitively, efficientnet-b2 and regnet_y_800mf are used as examples for detailed explanation. In the network architecture of efficientNet-b2, the fifth block is selected as the split point, and a feature extraction network is reconstructed from the first layer of the network to the dotted line split layer with the dotted line as the boundary. It is worth noting that the attention module (SE module) in efficientNet is cleverly integrated into the network structure in the form of a fully connected layer. In order to more comprehensively evaluate the impact of the SE module on network performance, the network performance after deleting the SE module is significantly improved compared to retaining the SE module.

[0068] Compared with the prior art, the template matching method of visual features provided in this embodiment uses a public image dataset to train a deep classification neural network model, improves the neural network structure to establish a target template matching model, can capture subtle differences and key information in the image, and thus improves the accuracy of template matching. By extracting features from the target template image dataset and the image to be matched, it can process a variety of image inputs, enhance the generalization ability in practical applications, and learn more generalized features in the image. The approximate cosine similarity of the template image features and the image features to be matched is calculated using the template image features as the convolution kernel, which can significantly improve the computational efficiency of matching. By setting a similarity threshold, the method can automatically screen out candidate images with a high similarity to the template image features during the matching process. The maximum pooling operation can retain the maximum similarity value in the local area, which helps to suppress the influence of noise and interference information on the matching results, thereby reducing the matching error rate.

[0069] See also Figure 3 , Figure 3 The present invention provides Figure 1In some embodiments of the present invention, extracting template image features and image features to be matched from a target template image dataset and an image to be matched by a target template matching model includes:

[0070] S301, establishing an initial template image data set, and preprocessing the initial template image data set to obtain a target template image data set;

[0071] S302, inputting the target template image data set into the target template matching model to extract multiple template image features;

[0072] S303: Input the image to be matched into the target template matching model to extract features of the image to be matched.

[0073] In the above embodiment, first, a large number of images related to the target application are collected from various sources (such as public databases, field photography, etc.) to form an initial template image data set. Then the data is cleaned to remove duplicate, blurred, incomplete or irrelevant images in the data set to ensure data quality. Preprocessing may include but is not limited to resizing, color correction, noise removal, contrast enhancement, etc. of the remaining images to standardize the image format and improve the accuracy of subsequent feature extraction. In addition, data enhancement techniques such as rotation, scaling, flipping, etc. may also be included to increase data diversity and enhance the generalization ability of the model. The preprocessed image set constitutes the target template image data set.

[0074] Select the appropriate template matching model according to the specific application requirements, which can include ResNet, EfficientNet and RegNet neural networks, or a combination of the two. Input the preprocessed target template image dataset into the selected model, and the model automatically learns and extracts multiple distinguishing image features by analyzing the image's texture, shape, color and other information. The extracted image features will be used in the subsequent matching process.

[0075] The image to be matched can also be subjected to similar preprocessing steps as those in the above preprocessing to ensure its consistency with the target template image dataset in terms of format and feature space. The preprocessed image to be matched is input into the trained target template matching model. Similarly, the model will extract the key features of the image to be matched. The extracted features of the image to be matched will be organized into a format suitable for comparison with the template image features, in preparation for the subsequent feature matching steps.

[0076] See also Figure 4 , Figure 4 The present invention provides Figure 3In some embodiments of the present invention, an initial template image data set is established, and the initial template image data set is preprocessed to obtain a target template image data set, including:

[0077] S401, cropping a plurality of basic template images to obtain an initial template image data set of a preset shape and a preset size;

[0078] S402, performing a rotation transformation on the initial template image data set to obtain a multi-angle template image data set;

[0079] S403 , performing multiple affine transformations on the multi-angle template image dataset to obtain a target template image dataset.

[0080] In the above embodiment, image processing software or programming tools are used to crop each basic template image according to actual application requirements to remove irrelevant background and redundant information and retain only the target area of ​​interest. The cropped image should have a uniform preset shape (such as square, circle, etc.) and preset size (such as 256x256 pixels, 128x128 pixels, etc.) to facilitate subsequent processing.

[0081] In order to enhance the diversity of template images and improve the matching algorithm's ability to recognize targets at different angles, the initial template image dataset is rotated. The rotation transformation can be performed around the center of the image or any specified point, and the rotation angle can be set as needed, such as rotating every 15 degrees, 30 degrees, or 45 degrees, to generate template images at multiple angles. Through the rotation transformation, a dataset containing template images at multiple angles is obtained, and these image datasets can more comprehensively reflect the appearance characteristics of the target at different viewing angles.

[0082] Affine transformation is a two-dimensional coordinate transformation, including translation, scaling, rotation (different from rotation transformation, further adjustment based on rotation), tilt and other operations. Various types of affine transformations are performed on the multi-angle template image dataset to simulate various deformations and perspective changes that the target may encounter in practical applications. The parameters of the affine transformation (such as translation, scaling, tilt angle, etc.) can be flexibly set according to the actual application requirements. By adjusting these parameters, a large number of template images with different deformations and perspective changes can be generated. After multiple affine transformations, a target template image dataset containing rich diversity and deformation information is obtained. This target template image dataset will be used to train the template matching model or as a reference template library in the matching process to improve the matching algorithm's recognition ability and robustness of the target under different conditions.

[0083] See also Figure 5 , Figure 5 The present invention provides Figure 1In some embodiments of the present invention, the simulation sliding window uses the template image feature as a convolution kernel to calculate the approximate cosine similarity between the template image feature and the image feature to be matched, including:

[0084] S501, using the template image feature as the convolution kernel and the image feature to be matched as the local vector to establish an initial approximate cosine similarity calculation formula;

[0085] S502, approximating the initial approximate cosine similarity calculation formula according to the template image features and the features of the image to be matched to obtain a target approximate cosine similarity calculation formula;

[0086] S503 , based on the target approximate cosine similarity calculation formula, respectively calculate the approximate cosine similarity between the image feature to be matched and each template image feature.

[0087] In the above embodiment, first, the template image features are regarded as convolution kernels (or filters), and the template image features are extracted by deep learning models (such as convolutional neural networks) and have a high degree of abstraction and representation capabilities. At the same time, the image features to be matched are divided into multiple local vectors (or local feature blocks), which are also obtained by the same deep learning model or other feature extraction methods. The division of local vectors can be determined according to the size of the image, the density of features, and the limitations of computing resources. Based on the convolution kernel (template image features) and local vectors (image features to be matched), an initial approximate cosine similarity calculation formula is constructed. This formula usually involves vector dot product operations and modulus length calculations, which are used to measure the degree of proximity between two vectors in direction.

[0088] Directly applying the initial approximate cosine similarity calculation formula may result in a huge amount of calculation, especially when processing high-resolution images or a large number of template images. Therefore, the formula needs to be approximated to reduce the computational complexity. The approximate processing method may include but is not limited to: using low-rank approximation to reduce the dimension of the feature vector; using fast Fourier transform (FFT) to accelerate the dot product operation; or using random projection and other techniques to approximate the vector modulus and dot product. After approximation, a target approximate cosine similarity calculation formula is obtained, which significantly reduces the computational complexity while maintaining the calculation accuracy.

[0089] For the image features to be matched, traverse each template image feature (i.e., each convolution kernel), and use the target approximate cosine similarity calculation formula to calculate the approximate cosine similarity between them. During the calculation process, the approximate cosine similarity between each template image feature and the image feature to be matched can be stored in a similarity matrix. Each row of this matrix represents a template image feature, and each column represents a local feature block of the image to be matched or a feature vector of the entire image (depending on how the local vector is divided). Finally, analyze the values ​​in the similarity matrix to find the template image feature that is most similar to the image feature to be matched, which involves finding the maximum value in the matrix or the row / column index where the maximum value is located, thereby determining the best matching result.

[0090] In some embodiments of the present invention, the initial approximate cosine similarity calculation formula is approximated according to the template image features and the image features to be matched to obtain the target approximate cosine similarity calculation formula, including:

[0091] The binary norm of the image feature to be matched in the initial approximate cosine similarity calculation formula is approximated to the binary norm of the template image feature, and a transitional approximate cosine similarity calculation formula is obtained;

[0092] The inner product of the unit vector of the template image feature and the feature of the image to be matched is used as the target approximate cosine similarity calculation formula.

[0093] In the above embodiment, approximate cosine similarity is adopted as the theoretical basis for measuring the similarity between vectors, as clearly shown in formula (1):

[0094]

[0095] Where A and B both represent vectors. In this formula framework, vector A corresponds to the convolution kernel (i.e., the template image feature), and vector B (i.e., the image feature to be matched) represents the local vector of the convolved tensor at the current window position. The symbol ||·|| represents the bi-norm of the vector,<A,B> represents the inner product operation of the vector. However, when the convolution operation is actually performed, the local calculation of each window position is essentially equivalent to the inner product operation of the corresponding local vector. Since the calculation of the two-norm of the local variable B is difficult to efficiently batch or parallelize, when vectors A and B are highly similar, their two-norms tend to be close. Based on this situation, the original formula (1) can be reasonably approximated to simplify the computational complexity:

[0096]

[0097] Specifically, the proposed new formula pre-calculates and stores the convolution kernel A's bi-norm, thereby avoiding the repeated calculation of B's ​​bi-norm each time the window slides. This improvement makes the new formula (2) significantly more efficient than the original formula (1) because it simplifies and accelerates the key calculation steps. Furthermore, in order to deepen this optimization strategy, formula (2) is further refined to ensure that the convolution operation can more closely simulate the target approximate cosine similarity calculation process under the sliding window:

[0098]

[0099] Through this series of innovative adjustments, not only the computational efficiency of the algorithm is improved, but also its practicality and scalability in processing large-scale data are enhanced.

[0100] See also Figure 6 , Figure 6 The present invention provides Figure 1 In some embodiments of the present invention, the template matching result is determined from the approximate cosine similarity according to the maximum pooling operation and the similarity threshold method, including:

[0101] S601, generating random numbers, and superimposing the random numbers to approximate cosine similarity;

[0102] S602, extracting a local maximum value from the superimposed approximate cosine similarities through a maximum pooling operation;

[0103] S603 : Determine a target template image that matches the image to be matched based on a similarity threshold method and a local maximum value.

[0104] In the above embodiment, in order to simulate the noise and uncertainty that may exist in practical applications, a series of random numbers can be generated and superimposed on the calculated approximate cosine similarity. These random numbers can follow normal distribution, uniform distribution or other suitable probability distribution, and the generated random numbers are superimposed element by element on the corresponding position of the approximate cosine similarity matrix, thereby introducing a small disturbance in the similarity matrix to simulate the noise and interference that may be encountered in practical applications. By introducing random numbers, the matching process can be made more robust to small similarity changes, reducing mismatches caused by noise or illumination changes.

[0105] Max pooling is a downsampling technique commonly used in convolutional neural networks. It reduces the dimensionality and redundancy of data by selecting the maximum value in a local region. In the template matching task, max pooling can be applied to the superimposed approximate cosine similarity matrix to extract the local maximum. The size of the local region can be determined based on the feature size of the template image and the image to be matched, as well as the limitations of computing resources. In general, the size of the local region should be large enough to cover the possible matching area, but not so large that detail information is lost. Within the defined local region, the maximum value of the approximate cosine similarity is selected as the representative value of the region. These local maxima will be used for subsequent matching decisions.

[0106] According to the needs of the actual application and background knowledge, a similarity threshold is set. This threshold is used to distinguish between successful matches and failed matches. The similarity threshold should be set high enough to exclude irrelevant template images, but not too high to miss potential matching objects. Compare the extracted local maximum with the set similarity threshold. If a local maximum is greater than or equal to the threshold, it is considered that the template image corresponding to the local area matches the image to be matched successfully. Otherwise, it is considered that the match fails. In the local area where the match is successful, select the template image with the highest similarity value as the final target template image.

[0107] See also Figure 7 , Figure 7 The present invention provides Figure 6 In some embodiments of the present invention, determining a target template image matching the image to be matched based on a similarity threshold method and a local maximum value includes:

[0108] S701, setting a similarity matching threshold according to historical template matching data;

[0109] S702, traversing all local maxima, and determining a local maximum value that exceeds a similarity matching threshold;

[0110] S703 , taking the template image corresponding to the maximum value among the local maximum values ​​exceeding the similarity matching threshold as the target template image.

[0111] In the above embodiment, first, historical template matching data is collected and analyzed. These data should include multiple groups of known successful matching template images and pairs of images to be matched, as well as the similarity values ​​between them. In addition, some known failed matching cases can also be included to understand the distribution of similarities in different situations. Based on historical data analysis, statistical methods (such as mean, median, standard deviation, etc.) can be used to set the similarity matching threshold. For example, a threshold value that is higher than the similarity value of most failed matching cases but lower than the similarity value of most successful matching cases can be selected. Another method is to use a machine learning algorithm (such as a support vector machine, a decision tree, etc.) to train a classifier to automatically determine the optimal similarity threshold. In practical applications, the similarity matching threshold may need to be dynamically adjusted based on the newly collected matching data. This can be achieved by regularly updating the historical database and recalculating the threshold.

[0112] In the previous step, local maxima have been extracted from the approximate cosine similarity matrix through the maximum pooling operation. Now, you need to traverse these local maxima to find those that exceed the similarity matching threshold. For each local maximum, compare it with the set similarity matching threshold. If the local maximum is greater than or equal to the threshold, it is considered a potential match and retained for subsequent analysis. Otherwise, it is excluded from the matching results. During the screening process, in addition to recording the local maxima that exceed the threshold, it is also necessary to record their corresponding template image index, location information, etc., so as to subsequently determine the target template image.

[0113] Among the local maxima that exceed the similarity matching threshold, the local maximum with the highest similarity value is selected. The local maximum represents the highest similarity between the image to be matched and a certain template image. According to the template image index or position information corresponding to the selected local maximum, the corresponding template image is retrieved from the template image database as the target template image. The target template image is the template image that best matches the image to be matched.

[0114] In order to better implement the template matching method of visual features in the embodiment of the present invention, based on the template matching method of visual features, correspondingly, please refer to Figure 8 , Figure 8 This is a schematic diagram of the structure of an embodiment of a visual feature template matching device provided by the present invention. The embodiment of the present invention provides a visual feature template matching device 800, including:

[0115] A model building module 810 is configured to train a deep classification neural network model using a public image dataset and improve the neural network structure to build a target template matching model;

[0116] A feature extraction module 820 is configured to extract template image features and to-be-matched image features from a target template image dataset and an to-be-matched image through a target template matching model;

[0117] A similarity calculation module 830 is configured to simulate a sliding window and use the template image feature as a convolution kernel to calculate the approximate cosine similarity between the template image feature and the image feature to be matched;

[0118] The template matching module 840 is configured to determine the template matching result from the approximate cosine similarity according to the maximum pooling operation and the similarity threshold method.

[0119] It should be noted here that: the device 800 provided in the above embodiment can implement the technical solutions described in the above method embodiments. The specific implementation principles of the above modules or units can refer to the corresponding contents in the above method embodiments, which will not be repeated here.

[0120] See also Fig. 9 , Fig. 9 A schematic diagram of the structure of a visual feature template matching device provided by an embodiment of the present invention. Based on the above visual feature template matching method, the present invention also provides a visual feature template matching device, which can be a computing device such as a mobile terminal, a desktop computer, a notebook, a palm computer, and a server. The visual feature template matching device 900 includes a processor 910, a memory 920, and a display 930. Fig. 9 Only some components of the visual feature template matching device are shown, but it should be understood that it is not required to implement all of the shown components, and more or fewer components may be implemented instead.

[0121] In some embodiments, the memory 920 may be an internal storage unit of the template matching device 900 of visual features, such as a hard disk or memory of the template matching device 900 of visual features. In other embodiments, the memory 920 may also be an external storage device of the template matching device 900 of visual features, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the template matching device 900 of visual features. Further, the memory 920 may also include both the internal storage unit of the template matching device 900 of visual features and an external storage device. The memory 920 is used to store application software and various types of data installed in the template matching device 900 of visual features, such as the program code of the template matching device 900 of visual features. The memory 920 may also be used to temporarily store data that has been output or is to be output. In one embodiment, a visual feature template matching program 940 is stored in the memory 920 , and the visual feature template matching program 940 can be executed by the processor 910 , thereby implementing the visual feature template matching method of each embodiment of the present application.

[0122] In some embodiments, the processor 910 may be a central processing unit (CPU), a microprocessor or other data processing chip, used to run program codes or process data stored in the memory 920, such as executing a template matching method of visual features.

[0123] In some embodiments, the display 930 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch device, etc. The display 930 is used to display information on the visual feature template matching device 900 and to display a visual user interface. The components 910-930 of the visual feature template matching device 900 communicate with each other via a system bus.

[0124] In one embodiment, when the processor 910 executes the visual feature template matching program 940 in the memory 920 , the steps in the visual feature template matching method described above are implemented.

[0125] This embodiment further provides a computer-readable storage medium on which a template matching program of visual features is stored. When the template matching program of visual features is executed by a processor, the following steps are implemented:

[0126] Use public image datasets to train deep classification neural network models, and improve the neural network structure to build a target template matching model;

[0127] Extracting template image features and image features to be matched from the target template image dataset and the image to be matched through a target template matching model;

[0128] The simulated sliding window uses the template image features as the convolution kernel to calculate the approximate cosine similarity between the template image features and the image features to be matched;

[0129] The template matching result is determined from the approximate cosine similarity based on the maximum pooling operation and the similarity threshold method.

[0130] In summary, the template matching method of visual features provided by the present invention can capture subtle differences and key information in images by using a public image dataset to train a deep classification neural network model, improve the neural network structure to establish a target template matching model, and thus improve the accuracy of template matching. By extracting features from the target template image dataset and the image to be matched, it can process a variety of image inputs, enhance the generalization ability in practical applications, and learn more generalized features in images. The approximate cosine similarity of the template image features and the image features to be matched is calculated using the template image features as the convolution kernel, which can significantly improve the calculation efficiency of matching. By setting a similarity threshold, the method can automatically screen out candidate images with high similarity to the template image features during the matching process. The maximum pooling operation can retain the maximum similarity value in the local area, which helps to suppress the influence of noise and interference information on the matching results, thereby reducing the matching error rate.

[0131] The present application also provides a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, the steps of the above method are implemented. The computer-readable storage medium may include, but is not limited to, any type of disk, including a floppy disk, an optical disk, a DVD, a CD-ROM, a micro drive, and a magneto-optical disk, a ROM, a RAM, an EPROM, an EEPROM, a DRAM, a VRAM, a flash memory device, a magnetic card or an optical card, a nanosystem (including a molecular memory IC), or any type of medium or device suitable for storing instructions and / or data.

[0132] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0133] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0134] In the several embodiments provided in the present application, it should be understood that the disclosed devices can be implemented in other ways. For example, the device embodiments described above are only schematic, such as the division of the units, which is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some service interfaces, and the indirect coupling or communication connection of devices or units can be electrical or other forms.

[0135] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0136] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0137] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a memory, including a number of instructions to enable a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned memory includes: U disk, read-only memory (ROM), random access memory (RAM), mobile hard disk, disk or optical disk and other media that can store program code.

[0138] A person of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable memory, which can include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0139] The above is only an exemplary embodiment of the present disclosure, and the scope of the present disclosure cannot be limited thereto. That is, any equivalent changes and modifications made according to the teachings of the present disclosure are still within the scope of the present disclosure. After considering the specification and practicing the disclosure here, those skilled in the art will easily think of the implementation scheme of the present disclosure. This application is intended to cover any modification, use or adaptation of the present disclosure, which follows the general principles of the present disclosure and includes common knowledge or customary technical means in the technical field not recorded in the present disclosure. The description and examples are regarded as exemplary only, and the scope and spirit of the present disclosure are defined by the claims.

[0140] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0141] It will be easily understood by those skilled in the art that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A template matching method of visual features, characterized in that: include: Use public image datasets to train deep classification neural network models, and improve the neural network structure to build a target template matching model; Extracting template image features and to-be-matched image features from the target template image dataset and the to-be-matched image through the target template matching model; The simulated sliding window uses the template image feature as a convolution kernel to calculate the approximate cosine similarity between the template image feature and the image feature to be matched; A template matching result is determined from the approximate cosine similarity according to a maximum pooling operation and a similarity threshold method.

2. The visual feature template matching method according to claim 1, characterized in that: The step of extracting template image features and image features to be matched from the target template image dataset and the image to be matched by using the target template matching model includes: Establishing an initial template image data set, and preprocessing the initial template image data set to obtain the target template image data set; Inputting the target template image data set into the target template matching model to extract multiple template image features; The image to be matched is input into the target template matching model to extract the features of the image to be matched.

3. The template matching method of visual features as claimed in claim 2, characterized in that: The step of establishing an initial template image data set and preprocessing the initial template image data set to obtain the target template image data set includes: Cropping a plurality of basic template images to obtain the initial template image data set of a preset shape and a preset size; Rotate the initial template image data set to obtain a multi-angle template image data set; The target template image dataset is obtained by performing a plurality of affine transformations on the multi-angle template image dataset.

4. The visual feature template matching method according to claim 1, characterized in that: The simulated sliding window uses the template image feature as a convolution kernel to calculate the approximate cosine similarity between the template image feature and the image feature to be matched, including: Using the template image feature as a convolution kernel and the image feature to be matched as a local vector to establish an initial approximate cosine similarity calculation formula; Approximating the initial approximate cosine similarity calculation formula according to the template image features and the to-be-matched image features to obtain a target approximate cosine similarity calculation formula; Based on the target approximate cosine similarity calculation formula, the approximate cosine similarity between the image feature to be matched and each of the template image features is calculated respectively.

5. The visual feature template matching method according to claim 4, characterized in that: The step of approximating the initial approximate cosine similarity calculation formula according to the template image features and the image features to be matched to obtain a target approximate cosine similarity calculation formula includes: Approximating the binary norm of the image feature to be matched in the initial approximate cosine similarity calculation formula to the binary norm of the template image feature to obtain a transitional approximate cosine similarity calculation formula; The inner product of the unit vector of the template image feature and the image feature to be matched is used as a target approximate cosine similarity calculation formula.

6. The visual feature template matching method according to claim 1, characterized in that: The determining the template matching result from the approximate cosine similarity according to the maximum pooling operation and the similarity threshold method includes: Generate a random number, and add the random number to the approximate cosine similarity; Extracting the local maximum from the superimposed approximate cosine similarity through the largest pooling operation; A target template image matching the image to be matched is determined based on the similarity threshold method and the local maximum value.

7. The visual feature template matching method according to claim 6, characterized in that: The determining of a target template image matching the image to be matched based on the similarity threshold method and the local maximum value includes: Set similarity matching threshold based on historical template matching data; Traversing all local maxima to determine a local maximum value that exceeds the similarity matching threshold; The template image corresponding to the maximum value among the local maximum values ​​exceeding the similarity matching threshold is used as the target template image.

8. A visual feature template matching device, characterized in that: include: A model building module, which is configured to train a deep classification neural network model using a public image dataset and improve the neural network structure to build a target template matching model; A feature extraction module, configured to extract template image features and to-be-matched image features from a target template image dataset and an image to be matched by using the target template matching model; A similarity calculation module is configured to simulate a sliding window and use the template image feature as a convolution kernel to calculate an approximate cosine similarity between the template image feature and the image feature to be matched; The template matching module is configured to determine a template matching result from the approximate cosine similarity according to a maximum pooling operation and a similarity threshold method.

9. A visual feature template matching device, characterized in that: It comprises at least one processing unit and at least one storage unit, wherein the storage unit stores a computer program, and when the computer program is executed by the processing unit, the processing unit executes the steps of the template matching method of visual features according to any one of claims 1 to 7.

10. A storage medium, characterized in that: It stores a computer program executable by an access authentication device. When the computer program runs on the access authentication device, the access authentication device executes the steps of the template matching method of visual features according to any one of claims 1 to 7.

Images