Feature point detection method, device, equipment and medium

Abstract

The invention discloses a feature point detection method and device, equipment and a medium. The method comprises the following steps: mapping an input image of a model into a high-dimensional feature vector; based on the high-dimensional feature vector, utilizing a first convolution operation to generate a position offset vector, a parameter corresponding to the first convolution operation being a model learnable parameter; determining a sampling position corresponding to the high-dimensional feature vector based on the position offset vector; sampling the high-dimensional feature vector by using the sampling position to obtain a sampling feature point; and determining a dynamic search area based on an area formed by the sampling feature points so as to perform feature point detection in the dynamic search area. Therefore, the position offset vector is dynamically changed along with model learning, the area formed by the sampled feature points obtained through sampling is also dynamic, the dynamic search area can be determined for feature point detection, the model can focus on important feature areas along with learning, and the accuracy of feature point detection is improved.

Description

Technical Field

[0001] The present application relates to the field of computer vision technology, and in particular to a feature point detection method, device, equipment and medium. Background Art

[0002] Feature point detection is one of the basic tasks in the field of computer vision. The goal is to automatically extract points with unique features from images. These feature points play a key role in subsequent visual tasks (such as target recognition, image matching, 3D reconstruction, etc.).

[0003] At present, for feature point detection, deep learning-based methods make full use of the expressive power of deep neural networks, automatically learn high-dimensional feature representations of images from large-scale data, and have strong robustness. However, due to the design of the network architecture, the focus area of ​​feature point detection is fixed, which may lead to insufficient information in key areas and affect the accuracy of feature point detection. Summary of the invention

[0004] In view of this, the purpose of this application is to provide a feature point detection method, device, equipment and medium, which can determine a dynamic search area for feature point detection, thereby improving the accuracy of feature point detection. The specific scheme is as follows:

[0005] In a first aspect, the present application discloses a feature point detection method, comprising:

[0006] Map the model’s input image into a high-dimensional feature vector;

[0007] Based on the high-dimensional feature vector, a position offset vector is generated by using a first convolution operation, wherein a parameter corresponding to the first convolution operation is a model learnable parameter;

[0008] Determine a sampling position corresponding to the high-dimensional feature vector based on the position offset vector;

[0009] Using the sampling position to sample the high-dimensional feature vector, to obtain sampling feature points;

[0010] A dynamic search area is determined based on the area composed of the sampled feature points to perform feature point detection within the dynamic search area.

[0011] Optionally, the generating a position offset vector based on the high-dimensional feature vector and using a first convolution operation includes:

[0012] Generate a query vector corresponding to the high-dimensional feature vector;

[0013] Downsampling the query vector to obtain a downsampled vector;

[0014] A first convolution operation is performed on the down-sampled vector to generate a position offset vector.

[0015] Optionally, determining the sampling position corresponding to the high-dimensional feature vector based on the position offset vector includes:

[0016] Performing a normalization operation on the position offset vector;

[0017] determining a scaling factor based on a spatial scale of the downsampled vector;

[0018] Using the scaling factor to adjust the position offset vector after the normalization operation to obtain an adjusted position offset vector;

[0019] A sampling position corresponding to the high-dimensional feature vector is determined based on the adjusted position offset vector.

[0020] Optionally, also include:

[0021] determining an original position based on a spatial scale of the downsampled vector;

[0022] Correspondingly, determining the sampling position corresponding to the high-dimensional feature vector based on the position offset vector includes:

[0023] The original position and the position offset vector are used to determine the offset position as the sampling position corresponding to the high-dimensional feature vector.

[0024] Optionally, performing feature point detection in the dynamic search area includes:

[0025] generating a key vector and a value vector using the dynamic search region;

[0026] Determine the feature point weights of the dynamic search area based on the down-sampling vector and the key vector;

[0027] Performing weighted calculation based on the feature point weights and the value vector to obtain a regional feature vector;

[0028] The target feature point coordinates are generated based on the region feature vector.

[0029] Optionally, generating target feature point coordinates based on the regional feature vector includes:

[0030] Generate a score vector using the regional feature vector, wherein the score vector includes a probability value of whether a target feature point exists in a sub-region in an input image of the model;

[0031] The target feature point coordinates are determined based on the probability value.

[0032] Optionally, also include:

[0033] Using a second convolution operation, setting the regional feature vector as a feature vector of a target dimension;

[0034] The target feature point coordinates are used to perform grid sampling on the feature vector of the target dimension to obtain a feature point descriptor.

[0035] In a second aspect, the present application discloses a feature point detection device, comprising:

[0036] A feature vector mapping module is used to map the input image of the model into a high-dimensional feature vector;

[0037] A position offset determination module, used to generate a position offset vector based on the high-dimensional feature vector and using a first convolution operation, wherein a parameter corresponding to the first convolution operation is a model learnable parameter;

[0038] A sampling position determination module, used to determine the sampling position corresponding to the high-dimensional feature vector based on the position offset vector;

[0039] A feature vector sampling module, used to sample the high-dimensional feature vector using the sampling position to obtain sampling feature points;

[0040] The search area determination module is used to determine a dynamic search area based on the area composed of the sampled feature points, so as to perform feature point detection in the dynamic search area.

[0041] In a third aspect, the present application discloses an electronic device, including a memory and a processor, wherein:

[0042] The memory is used to store the computer program;

[0043] The processor is used to execute the computer program to implement the aforementioned feature point detection method.

[0044] In a fourth aspect, the present application discloses a computer-readable storage medium for storing a computer program, wherein the computer program implements the aforementioned feature point detection method when executed by a processor.

[0045] It can be seen from the above scheme that the present application provides a feature point detection method, including: mapping the input image of the model into a high-dimensional feature vector; based on the high-dimensional feature vector, and using a first convolution operation to generate a position offset vector, wherein the parameters corresponding to the first convolution operation are model learnable parameters; based on the position offset vector, determining the sampling position corresponding to the high-dimensional feature vector; using the sampling position to sample the high-dimensional feature vector to obtain sampling feature points; determining a dynamic search area based on the area composed of the sampling feature points to perform feature point detection within the dynamic search area.

[0046] It can be seen that the beneficial effects of the present application are: mapping the input image of the model into a high-dimensional feature vector, the position offset vector used for sampling the high-dimensional feature vector is based on the high-dimensional feature vector and generated by the convolution operation, and the parameters corresponding to the convolution operation are the model learnable parameters, that is, the position offset vector changes dynamically as the model learns, and then the area formed by the sampled feature points obtained by sampling is also dynamic. In this way, the dynamic search area can be determined for feature point detection, so that the model can focus on important feature areas as it learns, thereby improving the accuracy of feature point detection.

[0047] Correspondingly, a feature point detection device, equipment and readable storage medium provided by the present application also have the above-mentioned technical effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0049] Figure 1 A flow chart of a feature point detection method provided in an embodiment of the present application;

[0050] Figure 2 A flow chart of dynamic search area feature point detection provided in an embodiment of the present application;

[0051] Figure 3 A flow chart of feature point coordinates and descriptor extraction provided in an embodiment of the present application;

[0052] Figure 4 A structural schematic diagram of a feature point detection device provided in an embodiment of the present application;

[0053] Figure 5 A structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0054] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0055] Feature point detection is one of the basic tasks in the field of computer vision. Its goal is to automatically extract points with unique features from images. These feature points play a key role in subsequent visual tasks (such as target recognition, image matching, 3D reconstruction, etc.). By accurately locating the feature points in the image, the visual system can perform robust image information matching without being affected by scale, lighting, angle, etc. Therefore, key point detection has been widely used in applications such as autonomous driving, augmented reality, and robot navigation.

[0056] In the research of feature point detection, there are two main categories: traditional methods and deep learning-based methods. Traditional methods extract image feature points by manually designing feature descriptors (such as SIFT[1], SURF[2], and ORB[3]), which are highly interpretable and intuitive. Such methods perform well in scenes with low resolution or high illumination changes and do not rely on large-scale datasets. However, the main disadvantage of traditional methods is that they are less sensitive to image content, cannot effectively cope with complex texture changes, and have limited expressiveness of feature descriptions, making it difficult to achieve higher matching accuracy and robustness.

[0057] Deep learning-based methods make full use of the expressive power of deep neural networks and can automatically learn high-dimensional feature representations of images from large-scale data. Such methods usually extract image features through convolutional neural networks (CNNs) and generate feature maps to identify feature points, such as network models such as SuperPoint[4], LF-Net[5], and D2-Net[6]. Compared with traditional methods, such methods have stronger robustness in various scenarios (such as indoors, outdoors, and from different perspectives). However, due to the simple design of the network architecture, the focus area of ​​feature point detection is fixed, which is difficult to dynamically adjust according to changes in image content, which may lead to insufficient information in key areas. Another common deep learning solution is to use a Transformer-based network to detect feature points in images. Transformer networks often require a large amount of computation to complete feature point detection, and the focus area for feature point detection is also relatively fixed, which greatly increases the difficulty of deploying the model on embedded devices. Therefore, it is of great significance to develop a feature point detection method that can dynamically search for key point areas while reducing computational complexity.

[0058] In summary, the existing methods (such as SuperPoint, D2-Net, LF-Net and Transformer network) focus on relatively fixed areas of feature points, which are difficult to dynamically adjust according to changes in image content, which may lead to insufficient information in key areas, thus seriously affecting the accuracy of feature point detection. In addition, the Transformer network also introduces a large amount of calculation, which limits the deployment of the model on embedded devices. In order to address the shortcomings of the existing methods, the present application provides a feature point detection solution based on a dynamic search area, which can dynamically adjust the feature point detection area and make full use of the features of the key areas of the image, thereby improving the performance of feature point detection in complex scenes. At the same time, through the dynamic search area strategy, the computational complexity of the model can be effectively reduced.

[0059] See also Figure 1 As shown, the embodiment of the present application discloses a feature point detection method, including:

[0060] Step S11: Mapping the input image of the model into a high-dimensional feature vector.

[0061] The input image of the model is the image input to the model, and the model is a deep learning network model. High-dimensional feature vector is a common concept in the field of machine learning and data analysis. A high-dimensional feature vector is a vector with a very high dimension (i.e., the number of features). That is, the dimension of the data far exceeds the three-dimensional space, and each feature can be regarded as a dimension.

[0062] In an optional implementation, the input image of the model can be mapped to a high-dimensional feature vector based on a preset convolution operation. For example, if the input image of the model is an RGB three-channel image, the image is mapped to a high-dimensional space with 64 channels through two 3×3 convolution operations to obtain a high-dimensional feature vector.

[0063] That is, the embodiment of the present application first transforms the input image into a high-dimensional space. For example, the input RGB three-channel image in, The spatial dimensions of the vector are three-dimensional, which are 3 (the number of channels of the image), H (the height of the image), and W (the width of the image). Two consecutive 3×3 convolution operations are used to map the image I to a high-dimensional space with 64 channels to obtain a high-dimensional feature vector X, where

[0064] X=Conv 3×3 (LN(GELU(Conv 3×3 (I)))) (1);

[0065] Among them, Conv 3×3(·) represents a 3×3 convolution operation, GELU(·) represents the GELU activation function, which is used to increase the nonlinearity of the feature; LN(·) represents the layer normalization function, which can be used to stabilize the distribution of feature data.

[0066] Step S12: Based on the high-dimensional feature vector, a position offset vector is generated by using a first convolution operation, wherein parameters corresponding to the first convolution operation are model learnable parameters.

[0067] The first convolution operation is a preset convolution operation for generating a position offset vector, and the position offset vector can be understood as a vector recording the position offset. The parameters corresponding to the first convolution operation are the parameters required for the first convolution operation, including convolution kernel parameters, etc. The model learns to update the parameters corresponding to the first convolution operation, and then updates the position offset vector.

[0068] In an optional implementation, a query vector corresponding to the high-dimensional feature vector may be generated; the query vector may be downsampled to obtain a downsampled vector; and a first convolution operation may be performed on the downsampled vector to generate a position offset vector.

[0069] In the embodiment of the present application, in the process of generating the position offset vector, the query vector is downsampled, which reduces the amount of data and can reduce the computational complexity of feature point detection.

[0070] It should be noted that, in this document, relational terms such as first and second, etc. are merely used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations.

[0071] Step S13: determining a sampling position corresponding to the high-dimensional feature vector based on the position offset vector.

[0072] In an optional implementation, the position offset vector may be normalized; a scaling factor may be determined based on the spatial scale of the downsampled vector; the position offset vector after the normalization operation may be adjusted using the scaling factor to obtain an adjusted position offset vector; and a sampling position corresponding to the high-dimensional feature vector may be determined based on the adjusted position offset vector.

[0073] The spatial scale is the height and width of the downsampling. In addition, the embodiment of the present application can also use a preset range limiting factor to control the absolute value range of the offset.

[0074] The embodiment of the present application can determine the original position based on the spatial scale of the down-sampled vector; accordingly, the offset position is determined using the original position and the position offset vector as the sampling position corresponding to the high-dimensional feature vector.

[0075] The original position is the position before the position shift, and the embodiment of the present application can use the original position plus the position shift vector to obtain the position after the shift. The original position can also be represented by a vector.

[0076] That is, the embodiment of the present application uses a high-dimensional feature vector to generate a position offset vector of a dynamic area, which may specifically include the following steps:

[0077] First, the obtained high-dimensional feature vector X is projected into the new Gaussian space to obtain the query vector Query (abbreviated as Q):

[0078] Q=Conv 1×1 (X) (2);

[0079] Among them, Conv 1×1 (·) represents a 1×1 convolution operation.

[0080] Then, a 7×7 convolution is used to extract the spatial position information of the query vector Q. In order to reduce the computational complexity, the query vector Q can be downsampled four times, that is, the step size of the 7×7 convolution is set to 4 and the padding number is set to 3, thereby reducing the query vector Q to one-fourth of the original, and obtaining the downsampled vector Q scale , the length and width of the space are H scale , W scale :

[0081] Q scale =LN(GELU(Conv 7×7 (Q))) (3);

[0082] Among them, Conv 7×7 (·) is a 7×7 convolution.

[0083] Then, a 1×1 convolution is used to change the number of channels of the downsampled vector to 2 (corresponding to the (x, y) coordinates of the offset). x and y can represent the position coordinates in the width direction and the height direction respectively, and the initial position offset vector offset is obtained. The elements in the position offset vector are offsets. Since the process of generating position offsets is implemented by convolution operations, the position offsets are a set of learnable parameters, and the model can dynamically update the position offset parameters. The process can be expressed as:

[0084] offset = Conv 1×1 (Q scale ) (4);

[0085] After obtaining the initial position offset vector, it is necessary to limit the value range of the position offset vector to ensure that each offset is within the preset value range. Specifically, the range limitation of the position offset is achieved in the following way:

[0086] (a) First normalize the initial position offset vector offset to the interval [-1,1]:

[0087] offset norm =tanh(offset) (5);

[0088] Among them, tanh(·) is the hyperbolic tangent activation function. Through the tanh(·) activation function, the range of the offset value can be limited, thereby avoiding the influence of extreme values ​​on subsequent calculations.

[0089] (b) Multiply the normalized offset vector by an appropriate scaling factor to ensure that the offset value is adjusted according to the spatial scale of the sample vector to accommodate changes in image size or resolution. By controlling the normalization factor, the offset can be constrained to a dynamic range to accommodate the diversity of image features:

[0090]

[0091] (c) Use the preset range limiting factor factor to further control the absolute value range of the offset, allowing the offset to be fine-tuned within a certain range to obtain the final position offset vector Δp ​​(here the range limiting factor can be set to 4):

[0092] Δp=offset scale factor (7);

[0093] In formula (6) and formula (7), · represents multiplication.

[0094] The embodiment of the present application may generate the original position using the spatial information of the downsampled query vector.

[0095] According to the spatial scale H of the downsampled query vector scale , W scale Generate the corresponding two-dimensional coordinates p(x origin ,y origin ), the generated x origin and origin Respectively represent the position of each point in the height and width directions. Finally, the generated original coordinates are normalized to the [-1,1] interval:

[0096]

[0097] Furthermore, based on the obtained position offset vector and the original position value, the coordinate pos after position offset can be obtained, that is, the sampling position.

[0098] Step S14: using the sampling position to sample the high-dimensional feature vector to obtain sampling feature points.

[0099] In an optional implementation, grid sampling may be performed on the high-dimensional feature vector to obtain sampling feature points. The sampling feature points are feature points obtained by sampling.

[0100] The embodiment of the present application can use the coordinates after position shift, i.e., the sampling position, to perform grid sampling on the high-dimensional feature vector. That is, according to the sampling points after the shift, feature points sampled after the shift are generated for the high-dimensional feature vector, and then these points form a dynamic search area. The grid sampling is implemented using bilinear interpolation:

[0101] X region =Grid_Sample(X,pos) (10);

[0102] Among them, Grid_Sample() is the grid sampling function.

[0103] Step S15: determining a dynamic search area based on the area composed of the sampled feature points, so as to perform feature point detection within the dynamic search area.

[0104] That is, the embodiment of the present application generates a dynamic search area using the position offset vector. The parameters of the first convolution operation are updated based on the model, the position offset vector is updated, and the search area determined is also dynamic.

[0105] In an optional embodiment, performing feature point detection within the dynamic search area may include: generating a key vector and a value vector using the dynamic search area; determining feature point weights of the dynamic search area based on the downsampling vector and the key vector; performing weighted calculation based on the feature point weights and the value vector to obtain a regional feature vector; and generating target feature point coordinates based on the regional feature vector.

[0106] Among them, a 1×1 convolution operation can be used to calculate the feature points of the dynamic search area to obtain a key vector and a value vector. The feature points of the dynamic search area are the feature points obtained by sampling. The dot product similarity is calculated between the key vector and the downsampled vector to obtain a weight that characterizes the importance of each feature point in the dynamic search area. That is, the embodiment of the present application performs a weighted calculation on the feature points in the dynamic search area, so that the model pays more attention to important features in the image.

[0107] In an optional embodiment, generating target feature point coordinates based on the regional feature vector includes: generating a score vector using the regional feature vector, wherein the score vector includes a probability value of whether a target feature point exists in a sub-region in the input image of the model; and determining the target feature point coordinates based on the probability value. It can be understood that the larger the probability value, the higher the probability that a feature point exists at the corresponding position. In the embodiment of the present application, the channel dimension of the weighted regional feature vector can be first set to a preset dimension based on a 1×1 convolution, and then a score vector can be generated based on a softmax activation function, and then non-maximum suppression algorithm filtering and boundary point filtering can be performed to obtain the target feature point coordinates.

[0108] Furthermore, the embodiment of the present application can use a second convolution operation to set the regional feature vector as a feature vector of a target dimension; and use the target feature point coordinates to perform grid sampling on the feature vector of the target dimension to obtain a feature point descriptor.

[0109] Among them, the second convolution operation can be understood as a step of processing the regional feature vector to obtain a feature vector of the target dimension. After obtaining the feature point descriptor, feature point matching between images can be performed to complete corresponding visual tasks, such as image matching, target tracking, etc. The embodiment of the present application can also perform a normalization operation on the feature point descriptor for subsequent processing.

[0110] For example, the embodiment of the present application uses the search area vector obtained by grid sampling, that is, the dynamic search area, to generate a key vector Key (abbreviated as K) and a value vector Value (abbreviated as V);

[0111] K=Conv 1x1 (X region ) (11);

[0112] V=Conv 1x1 (X region ) (12);

[0113] The K and V vectors are obtained based on the dynamic search area. scale , K vectors calculate the dot product similarity to obtain the importance weight of each feature point in the dynamic search area:

[0114] weight = Q scale ·K (13);

[0115] Finally, the feature points of the dynamic search area are weighted and summed to obtain the feature vector output F (i.e., regional feature vector) of the area:

[0116]

[0117] Where softmax(·) represents the softmax activation function, c represents the number of channels of the weight vector, and · in formula (13) and formula (14) represents the dot product.

[0118] Furthermore, the weighted feature vector is used to generate feature point coordinates and descriptors:

[0119] First, a 1×1 convolution is used to set the channel dimension of the weighted feature vector F to 65, and then a softmax activation function is used to obtain the score vector of whether there are feature points in F. The process can be expressed as:

[0120] score = softmax(Conv 1×1 (F)) (15);

[0121] The generated score vector has 65 channels, 64 of which represent the probability of the presence of a key point in an 8×8 grid region (i.e., a subregion) in the original input image, and the last channel represents the probability of the absence of a feature point in the region.

[0122] Then, the score matrix is ​​filtered using the non-maximum suppression algorithm (NMS), and the feature points at the image boundary are excluded. The final feature point Kp (i.e., the target feature point coordinates) is obtained according to the preset threshold. The preset threshold can be a quantity threshold, such as taking the 1000 points with the highest probability among the feature points after non-maximum suppression filtering and excluding the feature points at the boundary.

[0123] Secondly, use 1×1 convolution to set the channel dimension of the weighted feature vector F to 256, and obtain the feature vector descriptors used to generate feature point descriptors:

[0124] descriptors = Conv 1×1 (F) (16);

[0125] The obtained feature point Kp coordinates are normalized to the interval [-1,1], and the descriptors vector is grid sampled using the feature point coordinates to obtain the sampled feature point descriptor D:

[0126] D=Grid_Sample(descriptors,Kp) (17);

[0127] Finally, the sampled feature descriptor needs to be L2 normalized, that is:

[0128]

[0129] in, Represents the square operation of the cth channel of the D vector. out is the final normalized descriptor.

[0130] It can be seen that the embodiment of the present application maps the input image of the model into a high-dimensional feature vector. The position offset vector used for sampling the high-dimensional feature vector is based on the high-dimensional feature vector and is generated by a convolution operation. The parameters corresponding to the convolution operation are learnable model parameters, that is, the position offset vector changes dynamically as the model learns, and thus the area formed by the sampled feature points is also dynamic. In this way, a dynamic search area can be determined for feature point detection, so that the model can focus on important feature areas as it learns, thereby improving the accuracy of feature point detection.

[0131] For example, see Figure 2 As shown, Figure 2 A dynamic search area feature point detection flow chart provided in an embodiment of the present application. The input image uses 3×3 convolution to obtain a high-dimensional vector, performs 1×1 convolution and 7×7 convolution as well as GELU and LN functions to obtain a downsampled query vector, i.e., a downsampled vector, performs 1×1 convolution to obtain an initial position offset vector, normalizes the offset coordinates to obtain a position offset vector Δp, performs grid sampling, performs 1×1 convolution to obtain a key value, performs a dot product operation on the downsampled query vector and the key, and then performs weighted summation to extract feature point coordinates and descriptors to obtain feature point coordinates and descriptors.

[0132] See also Figure 3 As shown, Figure 3 A flow chart of feature point coordinates and descriptor extraction provided in an embodiment of the present application. The weighted feature vector is convolved by 1×1, softmax activated and NMS filtered to obtain feature point coordinates, grid sampling is performed based on the vector after 1×1 convolution and the feature point coordinates, and L2 normalization is performed to obtain the feature point descriptor.

[0133] The embodiment of the present application performs feature point detection based on a dynamic search area, and designs a mechanism for generating dynamic offset values. Through this mechanism, the model can dynamically generate an area for feature point detection, so that the model can focus on more important feature areas. In addition, downsampling is performed, and the calculation of the model can be limited to a small search area through a dynamic search area, thereby effectively reducing the computational complexity of the model. In this way, the search area for feature point detection is dynamically constructed, so that the model can focus on important feature areas, effectively improving the accuracy of feature point detection. By dynamically constructing the search area for feature point detection, the calculation range of the model can be effectively limited to the generated search area, thereby reducing the computational complexity of the model.

[0134] The feature point detection method provided in the embodiment of the present application can be applied to the model training process corresponding to the visual task. The model corresponding to the visual task is trained using an image sample set. Image samples in the image sample set can be obtained and input into the model corresponding to the visual task. The aforementioned feature point detection method is executed. After the model output is obtained, the model parameters are updated based on the model output, including determining the parameters of the first convolution operation of the position offset vector. After the sample image is input, the position offset vector is updated based on the updated parameters, and then the dynamic search area is determined, thereby realizing dynamic adjustment of the feature point detection area. Through learning, the model pays more attention to the feature point detection areas corresponding to more important feature points, thereby improving the accuracy of feature point detection and reducing computational complexity.

[0135] For further information, see Figure 4 As shown, the embodiment of the present application discloses a feature point detection device, including:

[0136] A feature vector mapping module 11, used for mapping the input image of the model into a high-dimensional feature vector;

[0137] A position offset determination module 12, configured to generate a position offset vector based on the high-dimensional feature vector and using a first convolution operation, wherein a parameter corresponding to the first convolution operation is a model learnable parameter;

[0138] A sampling position determination module 13, used to determine the sampling position corresponding to the high-dimensional feature vector based on the position offset vector;

[0139] A feature vector sampling module 14 is used to sample the high-dimensional feature vector using the sampling position to obtain sampling feature points;

[0140] The search area determination module 15 is used to determine a dynamic search area based on the area composed of the sampled feature points, so as to perform feature point detection in the dynamic search area.

[0141] In an optional implementation, the position offset determination module 12 may be used to generate a query vector corresponding to the high-dimensional feature vector; downsample the query vector to obtain a downsampled vector; and perform a first convolution operation on the downsampled vector to generate a position offset vector.

[0142] In an optional embodiment, the sampling position determination module 13 can be used to perform normalization on the position offset vector; determine the scaling factor based on the spatial scale of the down-sampling vector; use the scaling factor to adjust the position offset vector after the normalization operation to obtain an adjusted position offset vector; and determine the sampling position corresponding to the high-dimensional feature vector based on the adjusted position offset vector.

[0143] The device may further include:

[0144] An original position determination module, configured to determine an original position based on a spatial scale of the down-sampled vector;

[0145] Correspondingly, the sampling position determination module 13 can be used to determine the offset position using the original position and the position offset vector as the sampling position corresponding to the high-dimensional feature vector.

[0146] The device may further include a feature point detection module, and the feature point detection module may include:

[0147] A key-value generation submodule, used to generate a key vector and a value vector using the dynamic search area;

[0148] A weight determination submodule, used to determine the weight of the feature points of the dynamic search area based on the down-sampling vector and the key vector;

[0149] A feature weighting submodule, used for performing weighted calculation based on the feature point weights and the value vector to obtain a regional feature vector;

[0150] The feature point coordinate generation submodule is used to generate target feature point coordinates based on the regional feature vector.

[0151] Among them, the feature point coordinate generation submodule can be specifically used to generate a score vector using the regional feature vector, wherein the score vector includes a probability value of whether a target feature point exists in a sub-region in the input image of the model; and determine the target feature point coordinates based on the probability value.

[0152] Furthermore, the device may also include a feature point descriptor acquisition module, which is used to use a second convolution operation to set the regional feature vector as a feature vector of a target dimension; and use the target feature point coordinates to perform grid sampling on the feature vector of the target dimension to obtain a feature point descriptor.

[0153] It can be seen that the embodiment of the present application maps the input image of the model into a high-dimensional feature vector. The position offset vector used for sampling the high-dimensional feature vector is based on the high-dimensional feature vector and is generated by a convolution operation. The parameters corresponding to the convolution operation are learnable model parameters, that is, the position offset vector changes dynamically as the model learns, and thus the area formed by the sampled feature points is also dynamic. In this way, a dynamic search area can be determined for feature point detection, so that the model can focus on important feature areas as it learns, thereby improving the accuracy of feature point detection.

[0154] Furthermore, an embodiment of the present application discloses a computer program product, including a computer program / instruction, which, when executed by a processor, implements the feature point detection method disclosed in the aforementioned embodiment.

[0155] For the specific process of the above feature point detection method, reference may be made to the corresponding contents disclosed in the aforementioned embodiments, which will not be described in detail here.

[0156] See also Figure 5 As shown, an embodiment of the present application discloses an electronic device 20, including a processor 21 and a memory 22; wherein the memory 22 is used to store a computer program; and the processor 21 is used to execute the computer program, the feature point detection method disclosed in the above embodiment.

[0157] For the specific process of the above feature point detection method, reference may be made to the corresponding contents disclosed in the aforementioned embodiments, which will not be described in detail here.

[0158] Furthermore, the memory 22 as a carrier for storing resources may be a read-only memory, a random access memory, a magnetic disk or an optical disk, etc., and the storage method may be temporary storage or permanent storage.

[0159] In addition, the electronic device 20 also includes a power supply 23, a communication interface 24, an input / output interface 25 and a communication bus 26; wherein the power supply 23 is used to provide working voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and an external device, and the communication protocol it follows is any communication protocol that can be applied to the technical solution of the present application, and is not specifically limited here; the input / output interface 25 is used to obtain external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0160] Furthermore, an embodiment of the present application also discloses a computer-readable storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements the feature point detection method disclosed in the aforementioned embodiment.

[0161] For the specific process of the above feature point detection method, reference may be made to the corresponding contents disclosed in the aforementioned embodiments, which will not be described in detail here.

[0162] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0163] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0164] The above is a detailed introduction to a feature point detection method, device, equipment and medium provided by the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for general technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A feature point detection method, characterized in that: include: Map the model’s input image into a high-dimensional feature vector; Based on the high-dimensional feature vector, a position offset vector is generated by using a first convolution operation, wherein a parameter corresponding to the first convolution operation is a model learnable parameter; Determine a sampling position corresponding to the high-dimensional feature vector based on the position offset vector; Using the sampling position to sample the high-dimensional feature vector, to obtain sampling feature points; A dynamic search area is determined based on the area composed of the sampled feature points to perform feature point detection within the dynamic search area.

2. The feature point detection method according to claim 1, characterized in that: The step of generating a position offset vector based on the high-dimensional feature vector and using a first convolution operation includes: Generate a query vector corresponding to the high-dimensional feature vector; Downsampling the query vector to obtain a downsampled vector; A first convolution operation is performed on the down-sampled vector to generate a position offset vector.

3. The feature point detection method according to claim 2, characterized in that: The determining the sampling position corresponding to the high-dimensional feature vector based on the position offset vector includes: Performing a normalization operation on the position offset vector; determining a scaling factor based on a spatial scale of the downsampled vector; Using the scaling factor to adjust the position offset vector after the normalization operation to obtain an adjusted position offset vector; A sampling position corresponding to the high-dimensional feature vector is determined based on the adjusted position offset vector.

4. The feature point detection method according to claim 2, characterized in that: Also includes: determining an original position based on a spatial scale of the downsampled vector; Correspondingly, determining the sampling position corresponding to the high-dimensional feature vector based on the position offset vector includes: The original position and the position offset vector are used to determine the offset position as the sampling position corresponding to the high-dimensional feature vector.

5. The feature point detection method according to any one of claims 2 to 4, characterized in that: Performing feature point detection in the dynamic search area includes: generating a key vector and a value vector using the dynamic search region; Determine the feature point weights of the dynamic search area based on the down-sampling vector and the key vector; Performing weighted calculation based on the feature point weights and the value vector to obtain a regional feature vector; The target feature point coordinates are generated based on the region feature vector.

6. The feature point detection method according to claim 5, characterized in that: Generating target feature point coordinates based on the regional feature vector includes: Generate a score vector using the regional feature vector, wherein the score vector includes a probability value of whether a target feature point exists in a sub-region in an input image of the model; The target feature point coordinates are determined based on the probability value.

7. The feature point detection method according to claim 6, characterized in that: Also includes: Using a second convolution operation, setting the regional feature vector as a feature vector of a target dimension; The target feature point coordinates are used to perform grid sampling on the feature vector of the target dimension to obtain a feature point descriptor.

8. A feature point detection device, characterized in that: include: A feature vector mapping module is used to map the input image of the model into a high-dimensional feature vector; A position offset determination module, used to generate a position offset vector based on the high-dimensional feature vector and using a first convolution operation, wherein a parameter corresponding to the first convolution operation is a model learnable parameter; A sampling position determination module, used to determine the sampling position corresponding to the high-dimensional feature vector based on the position offset vector; A feature vector sampling module, used to sample the high-dimensional feature vector using the sampling position to obtain sampling feature points; The search area determination module is used to determine a dynamic search area based on the area composed of the sampled feature points, so as to perform feature point detection in the dynamic search area.

9. An electronic device, characterized in that: comprising a memory and a processor, wherein: The memory is used to store the computer program; The processor is used to execute the computer program to implement the feature point detection method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: Used to store a computer program, wherein when the computer program is executed by a processor, the feature point detection method according to any one of claims 1 to 7 is implemented.

Images