GreAtt feature description method

By establishing a feature point graph structure in feature point matching and using multi-layer perceptron to enhance the descriptor, the problems of missing and mismatch of feature point matching information in the prior art are solved, and more efficient and accurate feature point matching is achieved.

CN113762282BActive Publication Date: 2024-11-26CHENGDU QUANYI NETWORK TECHNOLOGY CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202110957083.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-08-19
Publication Date
2024-11-26
Estimated Expiration
2041-08-19

AI Technical Summary

Technical Problem

The existing feature description methods ignore the connection between adjacent feature points when matching feature points, resulting in missing information and mismatch, making it difficult to ensure the accuracy of the match, especially when the deformation field of biological tissue is complex.

Method used

The GreAtt feature description method is adopted to establish a graph structure by calculating the Euclidean distance and cosine distance between feature points, and map feature point position information and node information to high-dimensional space by using a multi-layer perceptron. Combining the SIFT algorithm and message delivery formula, it enhances the Information richness of feature descriptors.

Benefits of technology

It improves the accuracy and efficiency of feature point matching, reduces the number of wrong matching logs, enhances the information richness of the descriptor, and is suitable for the feature matching of biological tissue images in complex deformation fields.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113762282B_ABST
    Figure CN113762282B_ABST
Patent Text Reader

Abstract

The present invention relates to a GreAtt feature description method, comprising the following steps: 1. obtaining accurate position information and an initial feature descriptor of a feature point of an image; 2. calculating the Euclidean distance and cosine distance between the feature points corresponding to the initial feature descriptor respectively, and establishing connections between the feature points whose Euclidean distance and cosine distance meet a specific threshold value to form a feature point graph structure; 3. encoding the obtained feature point graph structure according to the horizontal coordinate and the vertical coordinate position from small to large and converting it into an adjacency matrix representation, the node degree in the graph structure is represented by a vector, and a multi-layer perceptron maps the position information and node information of the feature point to a high-dimensional space and combines them with the initial feature descriptor to obtain an enhanced feature descriptor. The present invention uses cosine distance and Euclidean distance to establish connections between feature points, which can avoid many invalid connections, thereby avoiding the problem of overly complex graph structure data and reducing excessive calculation input.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This patent belongs to the field of feature description methods, in particular to a GreAtt feature description method. Background Art

[0002] Most of the existing methods only consider the local feature information near the feature points when describing features, while ignoring the connection between adjacent feature points, which lacks information richness and leads to information loss in feature description. Due to the lack of description information and the more complex deformation field generated by biological tissue after being stressed, there are more pairs of feature point mismatches when performing image feature matching. The measurement accuracy of the flexible body deformation field is higher than that of image registration, so it is difficult for existing methods to ensure accurate matching of feature points. Summary of the invention

[0003] The technical problem to be solved by the present invention is to provide a GreAtt feature description method with accurate detection results and high efficiency.

[0004] The GreAtt feature description method includes the following steps:

[0005] Step 1: Obtain the precise location information and initial feature descriptors of the feature points of the image;

[0006] Step 2: Calculate the Euclidean distance and cosine distance between the feature points corresponding to the initial feature descriptors respectively, and establish connections between the feature points whose Euclidean distance and cosine distance meet specific thresholds to form a feature point graph structure;

[0007] Step 3: Encode the obtained feature point graph structure according to the horizontal and vertical coordinate positions from small to large and convert it into an adjacency matrix representation. The node degrees in the graph structure are represented by vectors. The multilayer perceptron maps the location information and node information of the feature points to a high-dimensional space and combines it with the initial feature descriptor to obtain an enhanced feature descriptor.

[0008] Furthermore, the step 2 specifically includes the following steps:

[0009] Step 2.1, obtaining the precise feature point position coordinates (x, y) of each initial feature descriptor in the initial feature descriptor extraction network;

[0010] Step 2.2, respectively calculate the Euclidean distance and cosine distance between the feature points. If the Euclidean distance between the two feature points is greater than 10 and the cosine distance is less than 0.3, then an edge is established between the two feature points, and finally the constructed feature point graph structure data N(G) can be obtained;

[0011] Furthermore, the step 3 specifically includes the following steps:

[0012] Step 3.1, sequentially encode the feature points in the image in descending order according to their horizontal and vertical coordinates, obtain the adjacency matrix of the feature point graph structure, and calculate the degree of each feature point and the characteristic root λ of the adjacency matrix;

[0013] Step 3.2: Use the formula to calculate the importance of each feature point Where Vj is the degree of the adjacent feature point, and α is the degree of the current feature point;

[0014] Step 3.3: Use a multi-layer perceptron to map the location information and importance of feature points into a high-dimensional space and combine it with the initial feature descriptor to obtain The message passing formula is used to propagate information along the edges in the feature point graph structure, and the transmitted information is aggregated and updated at different levels and between levels; the message propagation formula is: (K+1) x i = (K) x i +R([ (K) x i ||m ε→i ]), (K) x i ∈R D is the feature of feature point i on the Kth layer in the image, m ε→i is the result of the aggregation of messages of the connected feature points {j:(i,j)∈ε}, m ε→i The calculation formula is: ε→i =γ(x i ,x j ,ε j,i ), where γ represents a differentiable function, x i and x j Respectively represent the feature descriptors of feature points i and j, ε j,i Represents the characteristics of the edge from node j to node i;

[0015] Step 3.4: After all messages are aggregated, the feature descriptor corresponding to the feature point of the last layer is the enhanced feature descriptor f i .

[0016] Furthermore, in step 1, the SIFT algorithm is used to obtain the precise position information and initial feature descriptors of the feature points of the image.

[0017] The beneficial effects of the present invention are as follows: the present invention utilizes the machine learning method on the traditional feature extraction method, and innovatively combines the advantages of the two methods. Since the SIFT descriptor has a good description of local features, especially maintaining high stability and extensibility for rotation, scale scaling, and brightness changes, it is combined with the graph neural network, and at the same time draws on the selective visual attention mechanism in human vision, and uses the message passing method to transmit and aggregate the rich node information on the graph, and finally enriches the information of the descriptor.

[0018] In the construction method of graph structure data, cosine distance and Euclidean distance are used to establish connections between feature points. This is because the deformation field generated by biological tissue after being subjected to force will form multiple similar areas. This construction method also avoids many invalid connections, thereby avoiding the problem of overly complex graph structure data, reducing excessive computing input, and the detection results are accurate and efficient. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 It is a schematic diagram of the process of the present invention;

[0020] Figure 2 Extracting network diagrams for initial feature descriptors;

[0021] Figure 3 Schematic diagram of the process of constructing graph structure data for feature points;

[0022] Figure 4 These are the initial MR images of the calf and abdomen of a real volunteer;

[0023] Figure 5 For Figure 4 Corresponding initial MR images of the deformed calf and abdomen. DETAILED DESCRIPTION

[0024] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiment is a sub-embodiment of the present invention, rather than a full embodiment. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0025] like Figure 1 As shown, the overall technical solution of the present invention includes the following steps:

[0026] 1. Initial feature descriptor acquisition: First, use the SIFT algorithm to obtain the precise location information and initial feature descriptors of the feature points of the image.

[0027] 2. Construction of the image feature point graph structure: The cosine distance is used to calculate the distance of each feature descriptor combined with the position information and the Euclidean distance is used to calculate the distance of the feature points. Connections are established between feature points that meet a specific threshold to form a feature point graph structure.

[0028] 3. Attention focusing module: The obtained feature point graph structure is encoded from small to large according to the horizontal and vertical coordinate positions and converted into an adjacency matrix representation. The node degrees in the graph structure are represented by vectors. The multi-layer perceptron is used to map the location information and node information of the feature points to a high-dimensional space and combine them with the initial feature descriptor to obtain an enhanced feature descriptor.

[0029] 1. Initial feature descriptor extraction network

[0030] In the early 21st century, the Scale-Invariant Feature Transform (SIFT) algorithm represented by manual features was widely used in practical applications of image processing. However, the irregular and nonlinear deformation field formed by biological tissues after being subjected to force is greater and more complex than the deformation caused by nonlinear medical image registration such as organ peristalsis, breathing, and muscle contraction. In addition, the measurement accuracy requirements of the deformation field of flexible bodies are higher than those of image registration. Therefore, it is difficult to avoid the disadvantages brought by manual annotation of feature points and the problem of lack of feature expression.

[0031] However, because the SIFT algorithm has good invariance and strong scalability to translation, rotation, scale, brightness change, occlusion and noise, it can be easily combined with other forms of feature vectors. Therefore, in the first step, this method uses SIFT to extract image feature points x i and the initial feature descriptor d of the feature point i .

[0032] like Figure 2 As shown in the figure, in this network part, firstly, a 64*64 image is taken as input, and a Gaussian kernel function is used for filtering to construct a scale space, so that the original image can preserve the most detail features, thereby obtaining scale invariance; then, the DoG (Difference of Guassians) operator is used to detect the extreme values ​​of the constructed scale space and detect feature points; then, the principle that the principal curvature value is relatively large in the direction of the edge gradient and the principal curvature value is smaller along the edge direction is used to eliminate some useless feature points; finally, the gradient histogram statistics method is used to add feature descriptions to each feature point.

[0033] 2. Method for constructing image feature point graph structure

[0034] In the study of existing methods, especially in the use of graph neural networks, the data used are basically graph structure data. This paper makes innovations in the construction of graph structure data, combining the precise feature point position p obtained in the initial feature descriptor extraction network (x,y) and the initial feature descriptor d i To construct graph structure data.

[0035] By substituting the exact feature point position (x, y) into the Euclidean distance calculation formula:

[0036]

[0037] Among them, (x1, y1) represents the horizontal and vertical coordinates of one point, and (x2, y2) represents the horizontal and vertical coordinates of another point. i Substitute into the cosine distance calculation formula:

[0038]

[0039] Because, the feature descriptor d i is a high-dimensional vector, and a and b in the formula represent two vectors whose cosine distance needs to be calculated.

[0040] According to the size of the input image, the calculation results of the Euclidean distance and the cosine distance, this paper sets the two nodes with a Euclidean distance greater than 10 and a cosine distance less than 0.3 to establish an edge, so that the constructed feature point graph structure data can be finally obtained, and the feature points are sequentially encoded according to the order of the horizontal and vertical coordinates on the image from large to small to obtain the adjacency matrix of the graph structure, such as Figure 2 As shown in the figure, through this matrix, the program can know whether two nodes are connected. In the adjacency matrix, -1 means that there is no subsequent connection between the two nodes, and 1 means that the two nodes are connected. The degree of each feature point and the characteristic root λ of the adjacency matrix are calculated.

[0041] 3. Attention Focus Module

[0042] Considering the spatial and visual relationship of feature points in the same image, some useless feature points should be eliminated, and attention should be focused on the prominent feature points x i The information of the edge ε of the connected feature point can be used to calculate the feature point v by the following formula: j Importance I:

[0043]

[0044] Among them, Vj is the degree of the adjacent feature point, α is the degree of the feature point, and at the same time, the position information of the feature point, that is, the two-dimensional spatial position information obtained by the SIFT algorithm before, and the importance of the feature point are mapped to the high-dimensional space through the multi-layer perceptron (MLP), and combined with the initial feature vector to obtain The initial feature vector is the initial feature descriptor, and the same dimension is directly added. The message passing formula (4) is used to propagate information along the edges in the graph structure, and the transmitted information is aggregated and updated at different levels and between levels. For example, (K) x i ∈R D It can be regarded as the feature of feature point i on the Kth layer in the image. The result of the message passing formula m ε→i It is the result of message aggregation of the connected feature points {j:(i,j)∈ε}. The message passing formula is as follows:

[0045] (K+1) x i = (K) x i +R([ (K) x i ||m ε→i ]) (4)

[0046] Where m ε→i The calculation process is as follows:

[0047] m ε→i =γ(x i ,x j ,ε j,i ) (5)

[0048] γ represents a differentiable function, where x i , x j Respectively represent the feature descriptors of feature points i and j, ε j,i Represents the characteristics of the edge from node j to node i.

[0049] In each layer, all feature points can be learned and shared, so the descriptors obtained can be used to The result of message transmission between nodes is the enhanced feature descriptor f i , after the message aggregation is completed, the high-dimensional vector is obtained.

[0050] Example 1

[0051] Figure 4 and Figure 5The initial MR images of the calf and abdomen of a real volunteer and their corresponding deformed MR images. The equipment used is a 0.5T open MRI device with a FOV (scanning range) of 24*24CM and a slice spacing of 2mm.

[0052] Figure 4 In each pair of images, the left image is the MR image obtained in the original natural posture, and the corresponding right image is the MR image of biological tissue deformation under a certain pressure. Figure 4 (a) is an image of the abdomen deformed by thrust; (b) is an image of the abdomen deformed by pressure; Figure 4 (c) and Figure 4 (d) Deformation images of the abdomen of different volunteers when they were lying on their side and under pressure.

[0053] like Figure 5 As shown in the figure, the matching effect diagram obtained by using the nearest neighbor matching algorithm is shown. The feature descriptors in each pair of images are descriptors enhanced by the feature point descriptor method of the present invention, which can effectively reduce the number of false matching pairs. Compared with the existing enhanced feature descriptors based on the attention mechanism, Figure 5 In the experimental results, the number of wrong matching pairs has been reduced, especially in Figure 5 (a) In the experimental results of AbdPush, the number of mismatched pairs decreased by 1.6% year-on-year.

[0054] This paper uses the Euclidean distance of feature points and the cosine distance of feature descriptors to construct a graph data structure, which reduces the connection between similar feature points, simplifies the complexity of the graph data structure, and reduces unnecessary computational overhead for subsequent message passing. The importance of nodes is introduced in the process of message passing between nodes. In the process of message passing, this paper introduces the importance of nodes in the graph data structure, and passes rich information to each node.

[0055] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. GreAtt feature description method, characterized in that: The following steps are involved: Step 1: Obtain the precise location information and initial feature descriptors of the feature points of the image; Step 2: Calculate the Euclidean distance and cosine distance between the feature points corresponding to the initial feature descriptors respectively, and establish connections between the feature points whose Euclidean distance and cosine distance meet specific thresholds to form a feature point graph structure; Step 3: Encode the obtained feature point graph structure according to the horizontal and vertical coordinate positions from small to large and convert it into an adjacency matrix representation. The node degrees in the graph structure are represented by vectors. A multi-layer perceptron is used to map the location information and node information of the feature points to a high-dimensional space and combine them with the initial feature descriptor to obtain an enhanced feature descriptor. The step 3 specifically includes the following steps: Step 3.1, sequentially encode the feature points in the image in descending order according to their horizontal and vertical coordinates, obtain the adjacency matrix of the feature point graph structure, and calculate the degree of each feature point and the characteristic root λ of the adjacency matrix; Step 3.2: Use the formula to calculate the importance of each feature point Where Vj is the degree of the adjacent feature point, and α is the degree of the current feature point; Step 3.3: Use a multi-layer perceptron to map the location information and importance of feature points into a high-dimensional space and combine it with the initial feature descriptor to obtain The message passing formula is used to propagate information along the edges in the feature point graph structure, and the transmitted information is aggregated and updated at different levels and between levels; the message propagation formula is: (K+1) x i = (K) x i +R([ (K) x i ||m ε→i ]), (K) x i ∈R D is the feature of feature point i on the Kth layer in the image, m ε→i is the result of the aggregation of messages of the connected feature points {j:(i,j)∈ε}, m ε→i The calculation formula is: ε→i =γ(x i ,x j ,ε j,i ), where γ represents a differentiable function, x i and x j Respectively represent the feature descriptors of feature points i and j, ε j,i Represents the characteristics of the edge from node j to node i; Step 3.4: After all messages are aggregated, the feature descriptor corresponding to the feature point of the last layer is the enhanced feature descriptor f i .

2. The GreAtt feature description method according to claim 1, characterized in that: The step 2 specifically includes the following steps: Step 2.1, obtaining the precise feature point position coordinates (x, y) of each initial feature descriptor in the initial feature descriptor extraction network; Step 2.2, calculate the Euclidean distance and cosine distance between feature points respectively. If the Euclidean distance between two feature points is greater than 10 and the cosine distance is less than 0.3, an edge is established between the two feature points, and finally the constructed feature point graph structure data N(G) can be obtained.

3. The GreAtt feature description method according to claim 1, characterized in that: In step 1, the SIFT algorithm is used to obtain the precise position information and initial feature descriptors of the feature points of the image.