Deep learning feature point matching method, device and medium based on local receptive field

Abstract

The present invention provides a deep learning feature point matching method based on local receptive field. This method uses deep learning technology to construct a "lightweight" neural network model, which is more efficient and faster than other feature point quantization networks. Compared with traditional algorithms, it has higher matching accuracy under the same matching strategy. Therefore, this model takes into account both accuracy and speed, and has good practicality in industrial vision. The present invention also uses the characteristics of industrial 2D matching to limit the matching affine transformation matrix, namely, the scaling size limit and the deformation limit, which has good usability in actual production.

Description

Technical Field

[0001] The present invention relates to the technical field of neural networks, and in particular to a deep learning feature point matching method based on local receptive fields. Background Art

[0002] In the task of industrial 2D object matching, the algorithms used are generally divided into two categories. One is to screen and quantify the significant feature points of the object in the template image, and then calculate the distance of the quantized result vector for matching; the other is to scan the matched image in a way similar to image filtering to obtain the matching position.

[0003] Currently, the second category is still the most common among mature matching solutions. This is because compared to the first category, the second category is less prone to errors. However, since the matching process requires enumerating the angles and size scaling of the template object, this adds two dimensions to the search space. Even with the use of an image pyramid, this significantly increases the matching time. The first category, however, does not suffer from this problem because it is based on feature point matching. However, traditional feature point quantization methods are based on the characteristics of the feature point itself and the characteristics of a very small area. This results in the algorithm focusing on a very narrow range of feature points when extracting features. This narrow range of feature extraction can lead to errors during matching.

[0004] Therefore, how to improve the algorithm's focus on extracting feature points to improve the accuracy of matching is the first problem that needs to be faced and solved. Summary of the Invention

[0005] In response to the shortcomings of the existing technology, the present invention provides a deep learning feature point matching method, device and medium based on local receptive field to solve the technical problem that the existing technology has a narrow range of feature extraction, which easily leads to errors in matching.

[0006] The present invention provides a deep learning feature point matching method based on local receptive field, comprising:

[0007] S1. Build a neural network model, unify the input of the network model into pictures with a length and width of 32 pixels and a channel of 1, and classify the pictures into a training set and a test set;

[0008] S2, using the Triplet method to classify the training set into Data i 、Negative i and Positive i , where the Label i and Negative i For data with the same label, the Data i and Positivei The data is labeled oppositely, and the training set is fed into the network model;

[0009] S3, using the margin ranking loss function to train the network model, and using the feature point screening algorithm to screen out the feature points in the training set, and then intercepting the sub-image with a length and width of 32 pixels with the feature point as the center for training;

[0010] S4. Suppose that N and M feature points are selected from the template image and the search image respectively, and the vector representation results are obtained as follows: And calculate separately and The vector distances between each pair form an N×M matrix A N×M ;

[0011] S5. Set the first threshold Th of the vector distance to determine whether the matrix A N×M Each element A in ij Is it less than the first threshold Th? If so, record the coordinates of the template image and the coordinates of the search image to form a pair, generate multiple pairs of coordinates, and set the coordinate pair set as P;

[0012] S6. Consider the position mapping from the template image to the search image as an affine transformation, expressed as the affine transformation matrix M:

[0013]

[0014] S7. Randomly select three coordinate pairs from the set P, calculate the affine transformation matrix M of the coordinate pairs, and judge the affine transformation matrix M:

[0015] If the affine transformation matrix exists in M ​​with x1≠x5≠1 or x2≠-x4, it proves that the match is wrong. Otherwise, the coordinates of all template images in the set P are calculated by the affine transformation matrix M, and the second threshold Th is set. dist , and determine whether the distance between the result coordinates and the corresponding search graph coordinates in the set P is equal to the second threshold Th dist If the size is less than the second threshold Th dist , then the affine transformation of the position is considered correct, and a total of n correct points are obtained by counting, otherwise it is considered wrong;

[0016] S8. Repeat steps S6-S7 m times, and select the affine transformation matrix M with the largest number n of correct points as the final affine transformation matrix.

[0017] Optionally, constructing a neural network model includes:

[0018] The neural network model includes a first layer of 7×7 convolutional layer with a stride of 1 and 32 channels, a second layer of 2×2 max-pooling layer with a stride of 2, a third layer of 6×6 convolutional layer with a stride of 1 and 64 channels, and a fourth layer of a 128-dimensional fully connected layer.

[0019] Optionally, the adopting of the Margin ranking loss function to train the network model further includes:

[0020] The Margin ranking loss λ is expressed as:

[0021] λ(δ + ,δ - )=max(0,μ+δ + -δ - )

[0022] Among them, μ is a preset parameter, v + Represents the L2 distance between sample i and the positive sample:

[0023]

[0024] δ- represents the L2 distance between sample i and negative samples:

[0025]

[0026] Represented as Data i ,Negative i ,Positive i ,The vector table obtained through the neural network.

[0027] The present invention also provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the aforementioned method when executing the computer program.

[0028] The present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the steps of the aforementioned method when executed by a processor.

[0029] Compared with the prior art, the present invention has the following beneficial effects:

[0030] 1. Utilizing deep learning technology, we constructed a lightweight neural network model. Compared to other feature point quantization networks, this model is more efficient and faster. Compared to traditional algorithms, it achieves higher matching accuracy under the same matching strategy. This model balances both accuracy and speed, making it highly practical in industrial vision applications.

[0031] 2. This patent utilizes the characteristics of industrial 2D matching to restrict the matching affine transformation matrix, namely, the restriction of scaling size and the restriction of deformation, which has good usability in actual production. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0033] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0034] Figure 1 It is a schematic diagram of the process of the present invention;

[0035] Figure 2 Schematic diagram of the network structure in the present invention;

[0036] Figure 3 Schematic diagram of the comparison results between the model in the present invention and the SIFT model. DETAILED DESCRIPTION

[0037] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application. The functional units with the same labels in the examples of the present invention have the same and similar structures and functions.

[0038] See also Figure 1 The present invention provides a deep learning feature point matching method based on local receptive field, comprising:

[0039] S1. Build a neural network model, unify the input of the network model into pictures with a length and width of 32 pixels and a channel of 1, and classify the pictures into a training set and a test set;

[0040] S2, using the Triplet method to classify the training set into Data i 、Negative i and Positive i , where the Label iand Negative i For data with the same label, the Data i and Positive i The data is labeled oppositely, and the training set is fed into the network model;

[0041] S3, using the margin ranking loss function to train the network model, and using the feature point screening algorithm to screen out the feature points in the training set, and then intercepting the sub-image with a length and width of 32 pixels with the feature point as the center for training;

[0042] S4. Suppose that N and M feature points are selected from the template image and the search image respectively, and the vector representation results are obtained as follows: And calculate separately and The vector distances between each pair form an N×M matrix A N×M ;

[0043] S5. Set the first threshold Th of the vector distance to determine whether the matrix A N×M Each element A in ij Is it less than the first threshold Th? If so, record the coordinates of the template image and the coordinates of the search image to form a pair, generate multiple pairs of coordinates, and set the coordinate pair set as P;

[0044] S6. Consider the position mapping from the template image to the search image as an affine transformation, expressed as the affine transformation matrix M:

[0045]

[0046] S7. Randomly select three coordinate pairs from the set P, calculate the affine transformation matrix M of the coordinate pairs, and judge the affine transformation matrix M:

[0047] If the affine transformation matrix exists in M ​​with x1≠x5≠1 or x2≠-x4, it proves that the match is wrong. Otherwise, the coordinates of all template images in the set P are calculated by the affine transformation matrix M, and the second threshold Th is set. dist , and determine whether the distance between the result coordinates and the corresponding search graph coordinates in the set P is equal to the second threshold Th dist If the size is less than the second threshold Th dist , then the affine transformation of the position is considered correct, and a total of n correct points are obtained by counting, otherwise it is considered wrong.

[0048] S8. Repeat steps S6-S7 m times, and select the affine transformation matrix M with the largest number n of correct points as the final affine transformation matrix.

[0049] In this embodiment, S1, a neural network model is constructed, and the input of the network model is unified into an image with a length and width of 32 pixels and a channel of 1, and the image is classified into a training set and a test set. First, a lightweight neural network model is constructed. Under the premise of ensuring accuracy, compared with other feature point quantization networks, it is more efficient and faster, taking into account both accuracy and speed. It has good practicality in industrial vision. The input of the neural network model is unified into an image with a length of 32 pixels, a width of 32 pixels, and a channel of 1. The first layer is a 7×7 convolutional layer with a stride of 1 and a channel of 32. The second layer is a 2×2 max-pooling layer with a stride of 2. The third layer is a 6×6 convolutional layer with a stride of 1 and a channel of 64. The fourth layer is a 128-dimensional fully connected layer.

[0050] S2, using the Triplet method to classify the training set into Data i 、Negative i and Positive i , where the Label i and Negative i For data with the same label, the Data i and Positive i Negative is the data with opposite labels, and the training set is fed into the network model; i That is the data of the positive sample, Positive i That is the data of negative samples.

[0051] S3, using the margin ranking loss function to train the network model, and using the feature point screening algorithm to screen out the feature points in the training set, and then intercepting the sub-image with a length and width of 32 pixels with the feature point as the center for training;

[0052] See also Figure 2 , let Data i ,Negative i ,Positive i , the vector table obtained by the neural network is proved to be (That is, the 128-dimensional vector obtained after passing through the fully connected layer of the neural network)

[0053] Let δ + Represents the L2 distance between sample i and the positive sample,

[0054]

[0055] And δ -Represents the L2 distance between sample i and negative sample,

[0056]

[0057] Then the Margin ranking lossλ is expressed as:

[0058] λ(δ + ,δ - )=max(0,μ+δ + -δ - )

[0059] Among them, μ is a preset parameter. During training, the feature point screening algorithm is first used to filter out the feature points in the data set, and then the sub-image of the field is intercepted with this feature point as the center for training.

[0060] S4. Suppose that N and M feature points are selected from the template image and the search image respectively, and the vector representation results are obtained as follows: And calculate separately and The vector distances between each pair form an N×M matrix A N×M ;

[0061] S5. Set the first threshold Th of the vector distance to determine whether the matrix A N×M Each element A in ij Is it less than the first threshold Th? If so, record the coordinates of the template image and the coordinates of the search image to form a pair, generate multiple pairs of coordinates, and set the coordinate pair set as P;

[0062] S6. Consider the position mapping from the template image to the search image as an affine transformation, expressed as the affine transformation matrix M:

[0063]

[0064] S7. Randomly select three coordinate pairs from the set P, calculate the affine transformation matrix M of the coordinate pairs, and judge the affine transformation matrix M:

[0065] If the affine transformation matrix exists in M ​​with x1≠x5≠1 or x2≠-x4, it proves that the match is wrong. Otherwise, the coordinates of all template images in the set P are calculated by the affine transformation matrix M, and the second threshold Th is set. dist , and determine whether the distance between the result coordinates and the corresponding search graph coordinates in the set P is equal to the second threshold Th dist If the size is less than the second threshold Th dist , then the affine transformation of the position is considered correct, and a total of n correct points are obtained by counting, otherwise it is considered wrong.

[0066] S8. Repeat steps S6-S7 m times, and select the affine transformation matrix M with the largest number n of correct points as the final affine transformation matrix.

[0067] See also Figure 3 The figure shows the comparison between the model used in this paper and the SIFT model. The evaluation dataset used was the publicly available Photo Tour dataset, which is divided into three sub-datasets: Yosemite, Liberty, and Notredame. This model used one of these sub-datasets as the training set and the other as the test set to test the evaluation metrics. Six pairs of evaluation metrics were obtained, and the average of the metrics was calculated. Since SIFT does not require training, only one value was found for each test set.

[0068] The evaluation indicators in the table use the distance threshold when the true positive rate (TPR) of the calculated measurement model is 0.95, and then the false positive rate (FPR) of the model is calculated based on this threshold.

[0069] This paper utilizes deep learning technology to construct a lightweight neural network model that is more efficient and faster than other feature point quantization networks. Compared to traditional algorithms, it achieves higher matching accuracy under the same matching strategy. Therefore, this model balances both accuracy and speed, making it highly practical in industrial vision. Furthermore, leveraging the characteristics of industrial 2D matching, it imposes constraints on the matching affine transformation matrix, limiting both scaling and deformation, making it more user-friendly in actual production.

[0070] The present invention also provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the aforementioned method when executing the computer program.

[0071] The present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the steps of the aforementioned method when executed by a processor.

[0072] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device comprising the element.

[0073] The foregoing description is intended only to provide specific embodiments of the present invention, which will enable those skilled in the art to understand and implement the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention is not intended to be limited to the embodiments shown herein, but is intended to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A deep learning feature point matching method based on local receptive field, characterized in that: include: S1. Build a neural network model, unify the input of the network model into pictures with a length and width of 32 pixels and a channel of 1, and classify the pictures into a training set and a test set; S2, using the Triplet method to classify the training set into 、 Negative i and Positive i , and feed the training set into the network model; S3, adopt Margin ranking loss The network model is trained using a loss function, and a feature point screening algorithm is used to screen out feature points in the training set. Then, a sub-image with a length and width of 32 pixels is intercepted with the feature point as the center for training. Margin ranking loss The loss function trains the network model, including: described Margin ranking loss Expressed as: ； in, is a preset parameter. Representation sample i and the positive sample L2 distance: ； Representation sample i and negative samples L2 distance: ； Respectively expressed as , Negative i , Positive i, The vector table obtained by the neural network; S4, set to filter out the template image and search image respectively N and M feature points, and the vector representation results are , and calculate separately and The vector distance between two pairs, composed of Matrix ; S5. Set the first threshold of the vector distance Th , determine the matrix Each element in Is it less than the first threshold? Th If it is less than , record the coordinates of the template image and the search image to form a pair, generate multiple pairs of coordinates, and set the coordinate pair set as P ; S6. Consider the position mapping from the template image to the search image as an affine transformation, expressed as an affine transformation matrix M : ； S7, in the collection P Randomly select 3 pairs of coordinates and calculate the affine transformation matrix of the coordinate pairs M， Affine transformation matrix M Make a judgment: If the affine transformation matrix is M Existence or , then the matching is wrong, otherwise calculate the set again P The coordinates of all template images in the affine transformation matrix M The resulting coordinates of the transformation are then used to set the second threshold. , and judge the result coordinates and set P The distance between the corresponding search graph coordinates and the second threshold If the size is less than the second threshold , then the affine transformation of the position is considered correct, and the correct points are obtained statistically Otherwise it is considered wrong; S8. Repeat steps S6-S7 m times, select the correct number of points n Most affine transformation matrices M as the final affine transformation matrix.

2. The deep learning feature point matching method based on local receptive field according to claim 1, characterized in that The construction of the neural network model includes: The neural network model includes a first layer , a convolutional layer with a stride of 1 and a channel of 32, and the second layer is With a step size of 2 max-pooling The third layer is , a convolutional layer with a stride of 1 and a channel of 64, and the fourth layer is a 128-dimensional fully connected layer.

3. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 2 are implemented.

4. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 2 are implemented.

Images