A method for graph matching against noise-related

Abstract

The application provides a graph matching method for noise correlation, relates to the technical field of image matching, and comprises the following steps: initializing parameters of an online network and a momentum network, obtaining an input image and key points of the online network, inputting the online network to obtain key point representation of the online network, updating parameters of the momentum network, inputting the momentum network to obtain key point representation of the momentum network, calculating back propagation loss based on the key point representation of the online network and the key point representation of the momentum network to optimize the online network, repeatedly iterating until the online network converges to obtain node representation of the online network, obtaining a similarity matrix between nodes through the node representation of the online network, and calculating the similarity matrix through a Hungarian algorithm to obtain a key point matching matrix as a final matching result; the point alignment and edge alignment loss of the graph matching are calculated, the robustness of an image matching model to real noise correlation data is improved, and the accuracy of image matching is significantly improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image matching, in particular to a graph matching method for noise correlation. BACKGROUND

[0002] Given two images, the graph matching problem aims to establish the correct correspondence between the key points of the two images through the graph structure composed of the key points of the images. Graph matching has a wide range of application scenarios: through the correct establishment of the correspondence between the key points of different images, it can be applied to target tracking, three-dimensional reconstruction and other applications. The core of the graph matching method is to encode the graph structure, and the key is to explore and utilize the double-layer relationship in the graph structure to achieve accurate matching, i.e. the similarity relationship between nodes and the similarity relationship between edges. According to the way of encoding double-layer information, most of the existing methods can be divided into the following two categories: 1. Network design-based method, which implicitly aggregates information by designing a customized graph matching network; 2. Objective function design-based method, which explicitly learns double-layer information through different differentiable matching losses or optimization strategies.

[0003] At present, graph matching is widely used in various visual applications, such as three-dimensional reconstruction, target tracking, face recognition, etc. In three-dimensional reconstruction, through the feature point correspondence relationship established by the graph matching method, we can estimate the camera pose difference between different images, thereby splicing and fusing images of different perspectives to construct a three-dimensional scene. Although graph matching has achieved good performance in these fields, the current graph matching technology is highly dependent on perfect labeled alignment relationships. Existing graph matching methods all need to be guided by accurately aligned key points for training, but the labeling of key points in actual real data is extremely difficult. In real situations, there are often problems such as poor visibility of images, large differences in perspective between images, and occlusion of key points, and the manually labeled key points are usually not accurate enough, and even contain errors. The incorrectly labeled key points will produce incorrect matching relationships, which is called "noise correlation", which hinders the normal training of existing graph matching methods and ultimately leads to suboptimal and incorrect matching results. All current graph matching methods ignore the impact of noise correlation on network training, making it difficult to achieve optimal key point matching performance. At the same time, the noise correlation problem is inevitable: due to the complexity of the feature point labeling task (many key points, poor image visibility, etc.), it is difficult and unrealistic for humans to label a large number of accurate feature point correlations.

[0004] Based on this, the present application proposes a graph matching method for noise correlation to solve the above problems. SUMMARY

[0005] The present application aims to provide a graph matching method for noise correlation, which can improve the robustness of the image matching model to real noise correlation data and significantly improve the accuracy of image matching.

[0006] The technical scheme of the present application is:

[0007] In a first aspect, the present application provides a method for matching a graph associated with noise, comprising the following steps:

[0008] S1, initializing the parameters of an online network and a momentum network, and obtaining an input image and key points of the online network; wherein the online network and the momentum network are both composed of a VGG16 image encoder, a SplineCNN graph network encoder and a fully connected layer;

[0009] S2, inputting the input image and the key points into the online network to obtain the key point representation of the online network;

[0010] S3, updating the parameters of the momentum network based on the parameters of the online network;

[0011] S4, inputting the parameters of the momentum network into the momentum network to obtain the key point representation of the momentum network;

[0012] S5, calculating the backpropagation loss based on the key point representation of the online network and the key point representation of the momentum network, and optimizing the online network through the backpropagation loss;

[0013] S6, repeating steps S2-S5 until the online network converges to obtain the node representation of the online network, and obtaining the similarity matrix between nodes through the node representation of the online network;

[0014] S7, calculating the key point matching matrix through the Hungarian algorithm based on the similarity matrix to obtain the matching result.

[0015] Further, step S2 comprises:

[0016] S21, inputting the input image and the key points of the online network into the VGG16 image encoder of the online network to obtain initial node features;

[0017] S22, constructing the edge structure of the graph based on the key points through a triangulation algorithm, inputting the edge structure of the graph and the initial node features into the SplineCNN graph network encoder of the online network for optimization to obtain updated node features;

[0018] S23, inputting the updated node features into the fully connected layer of the online network to obtain the final node features as the key point representation of the online network.

[0019] Further, in step S21, the formula for inputting the input image and the key points of the online network into the VGG16 image encoder of the online network to obtain the initial node features is:

[0020]

[0021] where I A , I B denote two input images to be matched, denotes an image encoder of the online network f A , P B denotes the keypoint coordinates, Interp denotes a bilinear interpolation algorithm, U A , U B denotes the initial node features.

[0022] Further, in step S22, the formula for inputting the edge structure of the graph and the initial node features into the SplineCNN graph network encoder of the online network to obtain the updated node features is as follows:

[0023]

[0024] where denotes the updated node features, f splineCNN denotes a graph network encoder of the online network f A , E B denotes the edge structure of the graph, U A , U B denotes the initial node features.

[0025] Further, in step S23, the formula for inputting the updated node features into the fully connected layer of the online network to obtain the final node features is as follows:

[0026]

[0027] where denotes the updated node features, f MLP denotes a fully connected layer of the online network f A , V B denotes the final node features.

[0028] Further, in step S3, the formula for updating the parameters of the momentum network based on the parameters of the online network is as follows:

[0029] θ g ←tθ g +(1-t)θ f ,

[0030] where θ g denotes the parameters of the momentum network, θ f denotes the parameters of the online network, and t denotes the update ratio.

[0031] Further, in step S5, the process of calculating the back propagation loss includes calculating a point alignment loss and an edge alignment loss, wherein a formula for calculating the point alignment loss is:

[0032]

[0033]

[0034] wherein, represents the point alignment loss, represents a matrix row-level cross-entropy loss, I n represents a unit matrix, p represents a row-level Softmax activation function, a represents a fixed parameter, and [·] i represents the i-th row of a matrix, [·] j represents the j-th row of a matrix, V A , V B represents a key point representation, represents a key point representation obtained by a momentum network, T represents the transpose of a matrix, t represents a Softmax temperature coefficient, n is a positive integer, and exp represents an exponential function;

[0035] A formula for calculating the edge alignment loss is:

[0036]

[0037]

[0038] wherein, represents the edge alignment loss, represents a matrix dot product, W represents a matrix measuring the confidence of each edge, T represents the transpose of a matrix, V A , V B represents a key point representation, represents a key point representation obtained by a momentum network, represents a vector outer product, i and j represent the number of rows of a matrix, and [·] ij represents the element in the i-th row and the j-th column of a matrix, S is a replacement parameter, and F represents the Frobenius norm of a matrix;

[0039] A formula for calculating the back propagation loss is:

[0040]

[0041] wherein, represents the back propagation loss value, represents the point alignment loss, represents the edge alignment loss.

[0042] Further, in step S7, the formula for calculating the similarity matrix by the Hungarian algorithm is as follows:

[0043]

[0044] Y = Hungarian(S),

[0045] wherein S represents the similarity matrix, T represents the transpose of the matrix, V A , V B represents the key point representation, Y represents the key point matching matrix, and Hungarian represents the Hungarian algorithm.

[0046] In a second aspect, the present application provides an electronic device, characterized in that it comprises:

[0047] a memory for storing one or more programs;

[0048] a processor;

[0049] When the one or more programs are executed by the processor, the method for graph matching with noise correlation according to any one of the first aspect is implemented.

[0050] In a third aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method for graph matching with noise correlation according to any one of the first aspect.

[0051] Compared with the prior art, the present application has at least the following advantages or beneficial effects:

[0052] (1) The method for graph matching with noise correlation improves the robustness of the matching model to real noise correlation data and significantly improves the accuracy of image matching by applying robust point alignment and edge alignment loss for graph matching to solve the noise correlation problem in actual labeled data.

[0053] (2) The present application reduces the weight of noise correlation by momentum distillation, thereby improving the matching performance. The picture and key point position are input into the momentum network, and the momentum network can estimate the confidence of the points and edges to adjust the weight of the alignment in the matching loss, thereby adaptively balancing the training of the online network. The final key point matching result can better improve the accuracy of applications such as three-dimensional reconstruction, face recognition, and target tracking. BRIEF DESCRIPTION OF DRAWINGS

[0054] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments. It should be understood that the following drawings only show some of the embodiments of the present application, and therefore should not be regarded as a limitation on the scope, and for those skilled in the art, other related drawings can also be obtained without creative labor on the basis of these drawings.

[0055] Figure 1 A step diagram of a noise-related graph matching method of the present application;

[0056] Figure 2 A performance comparison diagram of the present application and BBGM and NGM;

[0057] Figure 3 A schematic structural block diagram of an electronic device.

[0058] Icons: 101, memory; 102, processor; 103, communication interface. DETAILED DESCRIPTION

[0059] In order to make the purposes, technical solutions and advantages of the embodiments of the present application more clear, the following will combine the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are some of the embodiments of the present application, but not all the embodiments. The components of the embodiments of the present application described and shown in the drawings can be arranged and designed in various different configurations.

[0060] Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the claimed present application, but only represents selected embodiments of the present application. All other embodiments obtained by those skilled in the art on the basis of the embodiments in the present application without creative labor are within the scope of protection of the present application.

[0061] It should be noted that: similar reference numerals and letters represent similar items in the following drawings, and therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings.

[0062] It should be noted that: in this paper, the term "comprising" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or includes elements inherent to such process, method, article or device. Without more limitations, the elements defined by the statement "comprising" do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0063] In the description of the present application, it should also be noted that unless otherwise explicitly specified and limited, the terms "set", "connected" should be understood broadly, for example, can be fixedly connected, can be detachably connected, or integrally connected; can be mechanically connected, can be electrically connected; can be directly connected, can be indirectly connected through an intermediate medium, and can be internal communication of two elements. For those skilled in the art, the specific meanings of the above terms in the present application can be understood according to the specific circumstances.

[0064] Some embodiments of the present application will be described in detail below with reference to the accompanying drawings. In the case of no conflict, each of the following embodiments and each feature in the embodiments can be combined with each other.

[0065] Embodiment 1

[0066] Please refer to Figure 1 , Figure 1 The steps of a graph matching method for noise correlation provided by an embodiment of the present application are shown in the figure.

[0067] In a first aspect, the present application provides a graph matching method for noise correlation, which comprises the following steps:

[0068] S1, initializing the parameters of the online network and the momentum network, and obtaining the input image and the key points of the online network; wherein the online network and the momentum network are both composed of a VGG16 image encoder, a SplineCNN graph network encoder and a fully connected layer;

[0069] S2, inputting the input image and the key points into the online network to obtain the key point representation of the online network;

[0070] S3, updating the parameters of the momentum network based on the parameters of the online network;

[0071] S4, inputting the parameters of the momentum network into the momentum network to obtain the key point representation of the momentum network;

[0072] S5, calculating the back propagation loss based on the key point representation of the online network and the key point representation of the momentum network, and optimizing the online network through the back propagation loss;

[0073] S6, repeating steps S2-S5 until the online network converges to obtain the node representation of the online network, and obtaining the similarity matrix between nodes through the node representation of the online network;

[0074] S7, calculating the key point matching matrix through the Hungarian algorithm based on the similarity matrix to obtain the matching result.

[0075] As a preferred embodiment, step S2 comprises:

[0076] S21, inputting the input image and the key point of the online network into a VGG16 image encoder of the online network to obtain initial node features;

[0077] S22, constructing edge structure of a graph based on the key point through a triangulation algorithm, inputting the edge structure of the graph and the initial node features into a SplineCNN graph network encoder of the online network for optimization to obtain updated node features;

[0078] S23, inputting the updated node features into a fully connected layer of the online network to obtain final node features as key point representation of the online network.

[0079] As a preferred implementation, in step S21, the formula for inputting the input image and the key point of the online network into a VGG16 image encoder of the online network to obtain initial node features is:

[0080]

[0081] wherein, I A , I B denote two input images to be matched, denotes an image encoder of the online network f, P A , P B denotes key point coordinates, Interp denotes a bilinear interpolation algorithm, U A , U B denotes initial node features.

[0082] As a preferred implementation, in step S22, the formula for inputting the edge structure of the graph and the initial node features into a SplineCNN graph network encoder of the online network for optimization to obtain updated node features is:

[0083]

[0084] wherein, denotes updated node features, f SplineCNN denotes a graph network encoder of the online network f, E A , E B denotes edge structure of the graph, U A , U B denotes initial node features.

[0085] As a preferred implementation, in step S23, the formula for inputting the updated node features into a fully connected layer of the online network to obtain final node features is:

[0086]

[0087] wherein, f represents the updated node features. MLP V represents the fully connected layer of the online network f. A V B This represents the characteristics of the final node.

[0088] In a preferred embodiment, in step S3, the formula for updating the parameters of the momentum network based on the parameters of the online network is:

[0089] θ g ←tθ g +(1-t)θ f ,

[0090] Where, θ g θ represents the parameters of the momentum network. f This represents the parameters of the online network, where t represents the update ratio.

[0091] In a preferred embodiment, step S5, calculating the backpropagation loss, includes calculating the point alignment loss and the edge alignment loss. The formula for calculating the point alignment loss is:

[0092]

[0093]

[0094] in, Indicates point alignment loss. I represents the cross-entropy loss at the matrix row level. n Let represent the identity matrix, ρ represent the row-level Softmax activation function, α represent a fixed parameter, and [·] i Let [·] represent the i-th row of the matrix. j V represents the j-th row of the matrix. A V B This indicates key points. The key point representation obtained from the momentum network is given by , T represents the transpose of the matrix, τ represents the Softmax temperature coefficient, n is a positive integer, and exp represents the exponential function.

[0095] The formula for calculating the edge alignment loss is:

[0096]

[0097]

[0098] in, Indicates the edge alignment loss. V represents the matrix dot product, W represents the matrix that measures the confidence of each edge, T represents the transpose of the matrix, and V represents the matrix transpose. A V B This indicates key points. keypoint representation obtained by the momentum network, denotes the vector outer product, i, j denote the number of rows of the matrix, [·] ij denotes the element of the i-th row and the j-th column of the matrix, S is a replacement parameter, and F denotes the Frobenius norm of the matrix;

[0099] The formula for calculating the back propagation loss is:

[0100]

[0101] wherein, denotes the back propagation loss value, denotes the point alignment loss, denotes the edge alignment loss.

[0102] As a preferred embodiment, in step S7, the formula for calculating the similarity matrix by the Hungarian algorithm is:

[0103]

[0104] Y = Hungarian (S),

[0105] wherein, S denotes the similarity matrix, T denotes the transpose of the matrix, V A , V B denotes the keypoint representation, Y denotes the keypoint matching matrix, and Hungarian denotes the Hungarian algorithm.

[0106] Example 2

[0107] In order to verify the superiority of the technical solution, the technical solution is compared with other seven kinds of graph matching technologies, i.e., graph matching network (GMN), cross-graph neural network (PCA), neural graph matching network (NGM), iterative cross-graph matching network (IPCA), channel independent embedding matching (CIE), black box graph matching optimizer (BBGM), and apparent structure robust graph matching (ASAR). The present application uses the Willow Object dataset which contains 256 pictures from five categories, and the index is accuracy (Accuracy).

[0108] The experimental results are as follows:

[0109] Comparison table of the present application and other graph matching technologies

[0110]

[0111]

[0112] From the table, we can see that the patent method has a very large improvement in accuracy (1.6 percentage points) compared to other graph matching methods.

[0113] To further explore the robustness of our method, we manually add partial noise associations on the Willow dataset. Specifically, we add a disturbance (s, theta) on the originally manually annotated key points, where s is the displacement amount, theta is the direction of the shift, and s and theta are sampled from a uniform distribution: The displacement amount s is further scaled by the size of the target detection box. The noise rate is defined as the percentage of key points that are disturbed, and experiments are conducted with a noise rate interval of 0.1.

[0114] Please refer to Figure 2 , Figure 2 The performance comparison chart of the present application and the two methods with the best performance, BBGM and NGM, is shown.

[0115] From the results in the figure, we can observe that:

[0116] First, the patent method is significantly better than the comparison methods in all noise rate settings.

[0117] Second, as the noise rate increases, the accuracy of the comparison methods decreases much more than the patent method.

[0118] Example 3

[0119] Please refer to Figure 3 , Figure 3 A schematic structural block diagram of an electronic device provided in Example 3 of the present application.

[0120] An electronic device includes a memory 101, a processor 102, and a communication interface 103, which are directly or indirectly electrically connected to each other to realize the transmission or interaction of data. For example, these elements can be electrically connected to each other through one or more communication buses or signal lines. The memory 101 can be used to store software programs and modules, and the processor 102 can execute various functional applications and data processing by executing the software programs and modules stored in the memory 101. The communication interface 103 can be used for signaling or data communication with other node devices.

[0121] The memory 101 can be, but is not limited to, a random access memory (RAM), a read only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), etc.

[0122] The processor 102 can be an integrated circuit chip with signal processing capability. The processor 102 can be a general purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; or can be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic device, discrete gate or transistor logic device, discrete hardware component.

[0123] It can be understood that the structure shown in the figure is only schematic, and the method for matching a graph associated with noise can further include more or less components than those shown in the figure, or have a different configuration from that shown in the figure. Each component shown in the figure can be implemented in hardware, software, or a combination thereof.

[0124] In the embodiments of the present application, it should be understood that the disclosed method can also be implemented by other ways. The embodiments described above are only schematic, for example, the flowcharts or block diagrams in the accompanying drawings show the possible implementation architectures, functions and operation of the system, method and computer program product according to the embodiments of the present application. In this regard, each block in the flowcharts or block diagrams can represent a module, a segment or a portion of code which comprises one or more executable instructions for implementing the specified logic function. It should also be noted that in some alternative implementations, the functions shown in the blocks can occur in a different order than that shown in the figures. For example, two blocks shown in succession can in fact be executed substantially concurrently or in the reverse order, depending on the functionality involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented by dedicated hardware-based systems which perform the specified functions or actions, or can be implemented by a combination of dedicated hardware-based systems and computer instructions.

[0125] In addition, the functional modules in each embodiment of the present application can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0126] If the functions are implemented in the form of software function modules and sold or used as an independent product, they can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application can be embodied in the form of a software product, and the computer software product is stored in a storage medium, and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0127] In summary, the embodiment of the present application provides a graph matching method for noise correlation, by initializing the parameters of the online network and the momentum network, obtaining the input image and the key points of the online network, then inputting the online network to obtain the key point representation of the online network, updating the parameters of the momentum network, and inputting the momentum network to obtain the key point representation of the momentum network, then calculating the back propagation loss based on the key point representation of the online network and the key point representation of the momentum network to optimize the online network, repeating iteration until the online network converges to obtain the node representation of the online network, finally obtaining the similarity matrix between nodes through the node representation of the online network, calculating the similarity matrix to obtain the key point matching matrix as the matching result; the present application improves the robustness of the image matching model to real noise correlation data by calculating the point alignment and edge alignment loss of graph matching, and significantly improves the accuracy of image matching.

[0128] The above only describes the preferred embodiments of the present application and is not intended to limit the present application. For those skilled in the art, the present application can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

[0129] It is obvious for those skilled in the art that the present application is not limited to the details of the above exemplary embodiments, and the present application can be implemented in other specific forms without departing from the spirit or essential characteristics of the present application. Therefore, the embodiments should be regarded as exemplary and non-limiting, and the scope of the present application is defined by the appended claims rather than the above description, and all changes falling within the meaning and scope of the equivalent elements of the claims are intended to be included in the present application. Any reference signs in the claims should not be regarded as limiting the claims.

Claims

1. A method for noise-related graph matching, characterized in that, The method comprises the following steps: S1, initializing parameters of an online network and a momentum network, and obtaining an input image and key points of the online network; wherein the online network and the momentum network are both composed of a VGG16 image encoder, a SplineCNN graph network encoder, and a fully connected layer; S2, inputting the input image and the key points into the online network to obtain key point representation of the online network; S3, updating parameters of the momentum network based on parameters of the online network; S4, inputting the parameters of the momentum network into the momentum network to obtain key point representation of the momentum network; S5, calculating a back propagation loss based on the key point representation of the online network and the key point representation of the momentum network, and optimizing the online network through the back propagation loss; S6, repeating steps S2-S5 until the online network converges to obtain node representation of the online network, and obtaining a similarity matrix between nodes through the node representation of the online network; S7, calculating a key point matching matrix through the Hungarian algorithm based on the similarity matrix to obtain a matching result.

2. The method of claim 1, wherein, Step S2 comprises: S21, inputting the input image and the key points of the online network into the VGG16 image encoder of the online network to obtain initial node features; S22, constructing edge structure of a graph based on the key points through a triangulation algorithm, inputting the edge structure of the graph and the initial node features into the SplineCNN graph network encoder of the online network for optimization to obtain updated node features; S23, inputting the updated node features into the fully connected layer of the online network to obtain final node features as the key point representation of the online network.

3. A graph matching method against noise association as claimed in claim 2, wherein, In step S21, the formula for inputting the input image and the key points of the online network into the VGG16 image encoder of the online network to obtain initial node features is: where I A , I B denotes two input images to be matched, denotes an image encoder on-line network f, P A , P B denotes key point coordinates, Interp denotes a bilinear interpolation algorithm, U A , U B denotes initial node features.

4. The method of claim 2, wherein, In step S22, the formula for inputting the edge structure of the graph and the initial node features into the SplineCNN graph network encoder of the online network for optimization to obtain updated node features is: wherein, denotes updated node features, f SplineCNN denotes a graph network encoder for online networks f, E A , E B denotes the edge structure of the graph, U A , U B denotes initial node features.

5. The method of claim 2, wherein, In step S23, the formula for inputting the updated node features into the fully connected layer of the online network to obtain final node features is: wherein, denotes the updated node features, f MLP denotes the fully connected layer of the online network f, V A , V B denotes the final node features.

6. The method of claim 1, wherein, In step S3, the formula for updating parameters of the momentum network based on parameters of the online network is: θ g ← t θ g + (1 - t) θ f , where θ g denotes the parameters of the momentum network, θ f denotes the parameters of the online network, and t denotes the update ratio.

7. The method of claim 1, wherein, In step S5, the process of calculating the back propagation loss comprises calculating a point alignment loss and an edge alignment loss, wherein the formula for calculating the point alignment loss is: wherein, represents the point alignment loss, represents the matrix row-level cross-entropy loss, I n represents the identity matrix, p represents the row-level Softmax activation function, a represents a fixed parameter, [·] i represents the i-th row of the matrix, [·] j represents the j-th row of the matrix, V A , V B represents the keypoint representation, represents the keypoint representation obtained by the momentum network, T represents the transpose of the matrix, t represents the Softmax temperature coefficient, n is a positive integer, and exp represents the exponential function. The formula for calculating the edge alignment loss is: wherein, denotes the edge alignment loss, denotes the matrix dot product, w denotes a matrix measuring the confidence of each edge, T denotes the transpose of the matrix, V A , V B denotes the keypoint representation, denotes the keypoint representation obtained by the momentum network, denotes the vector outer product, i, j denote the number of rows of the matrix, [·] ij denotes the element of the matrix in the i-th row and the j-th column, S is a replacement parameter, F denotes the Frobenius norm of the matrix; The formula for calculating the back propagation loss is: wherein, represents a backpropagation loss value, represents a point alignment loss, represents an edge alignment loss.

8. The method of claim 1, wherein, In step S7, the formula for calculating the similarity matrix through the Hungarian algorithm is: Y = Hungarian(S), where S denotes a similarity matrix, T denotes a transpose of a matrix, V A B denotes a keypoint representation, Y denotes a keypoint matching matrix, Hungarian denotes a Hungarian algorithm.​ 9. An electronic device, comprising: comprises: a memory for storing one or more programs; a processor; when the one or more programs are executed by the processor, a method for noise-related graph matching as claimed in any one of claims 1-8 is implemented.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement a method for noise-related graph matching as claimed in any one of claims 1-8.

Images