Large-scale deformation compensation methods, devices, equipment and media

Abstract

This invention discloses a method, apparatus, device, and medium for large-scale deformation compensation, relating to the field of image processing. This method treats salient points in an image as leaders within a group, similar to the alpha animal in a migrating flock, playing a crucial role in guiding the entire registration process. To estimate the clustering manifold, large-scale deformation registration is divided into three parts according to the animal's migration pattern: route decision based on clustering quantization, route execution with clustering preservation, and cascade migration with clustering inheritance. These three parts are integrated into a Clustering Cascade Migration (CCM) framework, effectively compensating for large-scale image deformation. This method can significantly improve the registration accuracy and robustness when compensating for large-scale deformation. This method is general, not limited to specific feature descriptors or deformation models, and can be easily extended to other image registration tasks.

Description

Technical Field

[0001] This invention relates to the field of image processing, and in particular to a method, apparatus, device, and medium for large-scale deformation compensation based on focus registration. Background Technology

[0002] Image registration is the process of matching and overlaying two or more images acquired at different times, by different sensors (imaging devices), or under different conditions (weather, illumination, camera position and angle, etc.). It has been widely used in remote sensing data analysis, computer vision, image processing and other fields.

[0003] Image registration and correlation is a typical problem and technical challenge in the field of image processing research. Its purpose is to compare or fuse images of the same object acquired under different conditions, such as images from different acquisition devices, at different times, and from different shooting angles. Sometimes, image registration for different objects is also required.

[0004] Specifically, for two images in an image dataset, a spatial transformation is found to map one image (moving image) onto the other image (fixed image), ensuring a one-to-one correspondence between points at the same spatial location in the two images, thus achieving information fusion. Widely used registration methods include image registration methods based on grayscale information and registration methods based on feature point matching.

[0005] However, traditional registration methods based on image grayscale similarity statistics have many drawbacks. For example, when faced with large-scale deformations, the objective function often becomes non-convex because it does not consider the role of salient points. Using traditional small update steps (usually used to optimize the objective function to ensure convergence stability) may cause the algorithm to get trapped in local minima. Traditional registration methods based on feature point matching also have many drawbacks; for example, locating reliable feature points is difficult, and balancing the quantity and quality of feature points is a challenge.

[0006] It is evident that when faced with large-scale deformation, existing technologies may be unable to accurately compensate for the deformation due to the lack of information on the relative displacement between pixels. Summary of the Invention

[0007] In view of the above problems, the present invention provides a method, apparatus, device and medium for large-scale deformation compensation to overcome the above problems or at least partially solve the above problems.

[0008] This invention provides the following solution:

[0009] A method for large-scale deformation compensation includes:

[0010] Feature extraction is performed on fixed and floating images to obtain feature extraction results. Then, a feature matching method and a parameter ratio test threshold are used in combination with the feature extraction results to identify several candidate routes. The candidate routes are the potential correspondence between feature points in the fixed image and feature points in the floating image.

[0011] A route consistency quantization module is used to quantify the consistency of several candidate routes to obtain a consistency matrix; the route consistency quantization module is used to perform direction consistency quantization, distance consistency quantization and topology consistency quantization on several candidate routes.

[0012] A clustering algorithm is used to determine several clustering routes from the consensus matrix;

[0013] A route checking strategy is employed to determine a final collective route from several clustered routes, the final collective route being used to drive large-scale deformation compensation of the image; the route checking strategy is also used to enhance the measurement of the similarity of local images in order to exclude routes that contribute negatively.

[0014] The flight field guiding the group is obtained by thin-plate spline interpolation of the final collective route, and the flight field is a large-scale dense deformation field;

[0015] The diffusion field is obtained by combining the route dwell strategy with the flight field estimation. The diffusion field is a small-scale dense deformation field.

[0016] The flight field and the diffusion field are combined to form a deformation field to obtain a migration field; the migration field is used to promote the deformation of the floating image, and to perform large-scale deformation compensation and small-scale deformation compensation on the floating image during the deformation process.

[0017] The floating image is continuously deformed through multiple migrations using a cascaded migration framework and the migration field until the migration converges, thus obtaining the target deformed image.

[0018] Preferably, the most lenient ratio test is used to determine all feature points with bijective matching as candidate routes.

[0019] Preferably, the distance consistency is represented by the following formula:

[0020]

[0021] In the formula: k dis ∈(0,+∞] is used to control the sensitivity of distance consistency d(i,j), d m d represents the relative distance in the floating image. fRepresents relative distances in a fixed image;

[0022] A voting method based on Avalon is used for topological consistency quantification, whereby topological consistency is represented by the following formula:

[0023] t(i,j) = t(i)·t(j)

[0024] In the formula: t(i) represents the route v i The route-level topology, t(j) represents route v j Route-level topology, route v j Indicates route v i The neighbor's route.

[0025] Preferably, t(i) is expressed by the following formula:

[0026]

[0027] In the formula: k top ∈(0,+∞] is used to control the sensitivity of the topology t(i), J rr (v i ) represents v i Compared to its neighbors Topological changes;

[0028] The J rr (v i It can be expressed by the following formula:

[0029]

[0030] In the formula: J without (v i ) and J with (v i ) respectively represent the introduction of v i The topology before and after.

[0031] Preferably, the final collective route is represented by the following formula:

[0032]

[0033] In the formula: c i Represents Ω i The contribution of the i-th collective route, Ω i This represents the i-th region of interest.

[0034] Preferably: the route stay strategy is a collective route. The drift distance, as a penalty term in the objective function, is expressed by the following formula:

[0035]

[0036] In the formula: Indicates normalized cross-correlation similarity. This indicates a stop on the route, defined as follows: k rs The parameter represents the trade-off between similarity and penalty. Indicates airfield, This represents the diffusion field.

[0037] Preferably: Before the cascaded migration converges, the current collective route is merged into the previous collective route. In, it is represented by the following formula:

[0038]

[0039] In the formula: Indicates the current collective route. This indicates the route taken by the previous group.

[0040] A large-scale deformation compensation device, characterized in that it is used to perform the above-described large-scale deformation compensation method, the device comprising:

[0041] The candidate route identification unit is used to extract features from fixed images and floating images to obtain feature extraction results, and to identify several candidate routes by using a feature matching method and a parameter ratio test threshold in combination with the feature extraction results. The candidate routes are the potential correspondence between feature points in the fixed image and feature points in the floating image.

[0042] The consistency matrix acquisition unit is used to quantify the consistency of several candidate routes using the route consistency quantization module to obtain a consistency matrix; the route consistency quantization module is used to perform direction consistency quantization, distance consistency quantization and topology consistency quantization on several candidate routes.

[0043] A clustering route determination unit is used to determine several clustering routes from the consistency matrix using a clustering clustering algorithm;

[0044] The final collective route determination unit is used to determine a final collective route from several said clustered routes using a route checking strategy. The final collective route is used to drive large-scale deformation compensation of the image. The route checking strategy is used to enhance the measurement of the similarity of local images in order to exclude negatively contributing routes.

[0045] The flight field acquisition unit is used to obtain the flight field of the guiding group by thin plate spline interpolation of the final collective route. The flight field is a large-scale dense deformation field.

[0046] The diffusion field acquisition unit is used to obtain the diffusion field by combining the route dwelling strategy with the flight field estimation. The diffusion field is a small-scale dense deformation field.

[0047] The migration field acquisition unit is used to combine the flight field and the diffusion field into a deformation field to obtain a migration field; the migration field is used to promote the deformation of the floating image, and to perform large-scale deformation compensation and small-scale deformation compensation on the floating image during the deformation process.

[0048] A cascaded migration unit is used to continuously deform the floating image through multiple migrations using a cascaded migration framework and the migration field until the migration converges and the target deformed image is obtained.

[0049] A large-scale deformation compensation device, the device comprising a processor and a memory:

[0050] The memory is used to store program code and transmit the program code to the processor;

[0051] The processor is used to execute the large-scale deformation compensation method described above according to the instructions in the program code.

[0052] A computer-readable storage medium for storing program code for performing the above-described large-scale deformation compensation method.

[0053] According to specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0054] This application provides a method, apparatus, device, and medium for large-scale deformation compensation. The method treats salient points in an image as leaders within a group, similar to the alpha animal in a migrating flock, playing a crucial role in guiding the entire registration process. To estimate the clustering manifold, large-scale deformation registration is divided into three parts according to the animal's migration pattern: route decision based on clustering quantization, route execution with clustering preservation, and cascade migration with clustering inheritance. These three parts are integrated into a Clustering Cascade Migration (CCM) framework, effectively compensating for large-scale image deformation. This method significantly improves registration accuracy and robustness when compensating for large-scale deformation. This method is general-purpose, not limited to specific feature descriptors or deformation models, and can be easily extended to other image registration tasks. Because this method utilizes the concept of clustering motion to provide a new approach to the large-scale deformation problem, it will bring more scalability to the field of image registration.

[0055] Of course, any product implementing this invention does not necessarily need to achieve all of the advantages described above at the same time. Attached Figure Description

[0056] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly described below. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0057] Figure 1 This is a flowchart of the large-scale deformation compensation method provided in the embodiments of the present invention;

[0058] Figure 2 This is a framework diagram of the large-scale deformation compensation method provided in the embodiments of the present invention;

[0059] Figure 3 This is a schematic diagram of distance consistency provided in an embodiment of the present invention;

[0060] Figure 4 This is a schematic diagram illustrating high directional consistency and high distance consistency provided in an embodiment of the present invention;

[0061] Figure 5 This is a flowchart of the voting process based on Avalon provided in an embodiment of the present invention;

[0062] Figure 6 This is a schematic diagram of the large-scale deformation compensation device provided in an embodiment of the present invention;

[0063] Figure 7 This is a schematic diagram of the large-scale deformation compensation device provided in an embodiment of the present invention. Detailed Implementation

[0064] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention are within the scope of protection of the present invention.

[0065] See Figure 1 This invention provides a method for large-scale deformation compensation, such as... Figure 1 As shown, the method may include:

[0066] S101: Feature extraction is performed on the fixed image and the floating image to obtain feature extraction results. A feature matching method and a parameter ratio test threshold are used in combination with the feature extraction results to identify several candidate routes. The candidate routes are the potential correspondence between feature points in the fixed image and feature points in the floating image. In specific implementation, the embodiments of this application can provide the use of the most lenient ratio test to determine all feature points with bijective matching as the candidate routes.

[0067] S102: The consistency of several candidate routes is quantified using a route consistency quantization module to obtain a consistency matrix; the route consistency quantization module is used to perform direction consistency quantization, distance consistency quantization, and topology consistency quantization on several candidate routes; specifically, in this application embodiment, the distance consistency can be represented by the following formula:

[0068]

[0069] In the formula: k dis ∈(0,+∞] is used to control the sensitivity of distance consistency d(i,j), d m d represents the relative distance in the floating image. f Represents relative distances in a fixed image;

[0070] A voting method based on Avalon is used for topological consistency quantification, whereby topological consistency is represented by the following formula:

[0071] t(i,j) = t(i)·t(j)

[0072] In the formula: t(i) represents the route v i The route-level topology, t(j) represents route v j Route-level topology, route v j Indicates route v i The neighbor's route.

[0073] t(i) is expressed by the following formula:

[0074]

[0075] In the formula: k top ∈(0, +∞) is used to control the sensitivity of the topology t(i), J rr (vi) means v i Compared to its neighbors Topological changes;

[0076] The J rr (v i It can be expressed by the following formula:

[0077]

[0078] In the formula: J without (v i ) and J with (v i ) respectively represent the introduction of v i The topology before and after.

[0079] S103: Several clustering routes are determined from the consensus matrix using a clustering algorithm;

[0080] S104: A route checking strategy is used to determine a final collective route from among several clustered routes. The final collective route is used to drive large-scale deformation compensation of the image. The route checking strategy is used to enhance the measurement of the similarity of local images in order to exclude routes that contribute negatively. In a specific implementation, the embodiments of this application can provide that the final collective route is represented by the following formula:

[0081]

[0082] In the formula: c i Represents Ω i The contribution of the i-th collective route, Ω i This represents the i-th region of interest.

[0083] S105: The flight field guiding the group is obtained by thin-plate spline interpolation of the final collective route, and the flight field is a large-scale dense deformation field.

[0084] S106: The diffusion field is obtained by combining the route dwelling strategy with the flight field estimation. The diffusion field is a small-scale dense deformation field. In specific implementation, the embodiments of this application can provide the collective route of the route dwelling strategy. The drift distance, as a penalty term in the objective function, is expressed by the following formula:

[0085]

[0086] In the formula: Indicates normalized cross-correlation similarity. This indicates a stop on the route, defined as follows: k rs The parameter represents the trade-off between similarity and penalty. Indicates airfield, Indicates the diffusion field.

[0087] S107: Combine the flight field and the diffusion field with a deformation field to obtain a migration field; the migration field is used to promote the deformation of the floating image, and to perform large-scale deformation compensation and small-scale deformation compensation on the floating image during the deformation process.

[0088] S108: Using a cascaded migration framework and the migration field, the floating image is continuously deformed through multiple migrations until the migration converges to obtain the target deformed image.

[0089] Furthermore, before the cascaded migration converges, the current collective route is merged into the previous collective route. In, it is represented by the following formula:

[0090]

[0091] In the formula: Indicates the current collective route. This indicates the route taken by the previous group.

[0092] The large-scale deformation compensation method provided in this application modeles large deformation compensation as clustering motion, comprising three main components: route decision based on clustering quantization, route execution with clustering preservation, and cascading migration with clustering inheritance. In the first component, given fixed and floating images, a traditional feature matching method is used to identify all correspondences as candidate routes using a loose ratio test threshold.

[0093] Subsequently, combining directional consistency, distance consistency and topological consistency are used to quantify the consistency of these routes. An efficient Avalon-based voting method is designed to compute topological consistency. Based on the consistency matrix, route clustering is applied to derive the collective routes. Finally, a route checking strategy is proposed to determine the final route by excluding routes that contribute negatively, demonstrating its ability to drive large deformation compensation in images.

[0094] In the second component, the large-scale dense deformation field of the guiding group is obtained using TPS interpolation of the final route, termed "flight". Then, under the initialization of the flight, the small-scale dense deformation field of the image grayscale is estimated, termed "diffusion". To maintain the clustering of the route, a route-staying strategy is proposed, keeping the leader point on the original route while allowing the group to diffuse. Flight and diffusion together constitute a migration that facilitates the transformation of the floating image.

[0095] In the third component, a cascading migration framework with a route merging strategy is introduced, transferring the current route to the next migration while ensuring consistency between them. When a new route cannot be identified in the current iteration, the framework converges, resulting in the final deformed image.

[0096] The large-scale deformation compensation method provided in this application will be described in detail below.

[0097] like Figure 2 As shown, after quantifying and maintaining the clustering of routes, collective migration facilitates smooth transformations, compensating for both large-scale and small-scale deformations. Multiple migrations are integrated into a cascading migration framework until convergence, i.e., when no new routes can be identified in the current iteration.

[0098] Route decision based on cluster quantification.

[0099] Candidate routes have been determined.

[0100] Candidate routes are defined as feature points p in a fixed image f and a floating image m.f and p m The potential correspondences between them can be identified using feature matching methods. In this application, FAST+BRISK+mutual nearest neighbor search is used to identify candidate routes. This method can perform feature extraction and description quickly and efficiently on various datasets without requiring a large amount of labeled datasets for training. For 3D datasets, extending FAST and BRISK to 3D scenes is a trivial extension. This extension can be efficiently implemented using parallel computing techniques on GPUs.

[0101] The ratio test (r) controls the number of candidate routes. In this application, the most lenient ratio test (r = 1) is used to include all feature points with bijective matching as candidate routes. This approach contrasts with many marker-based methods, which require guaranteed matching accuracy and typically use only the top 1% of matches and Random Sample Consensus (RANSAC) techniques; otherwise, the accuracy of the estimated deformation field drops drastically. In this application, correct matches are extracted based on route clustering, allowing all matches to be used as input without sacrificing accuracy. On the contrary, accuracy is improved due to increased match recall.

[0102] Route consistency quantification.

[0103] After determining the candidate routes, the set of leader points P can be obtained. f and its destination set P m and the set of routes between them, V = {p m -p f |p f ∈P f ,p m ∈P m The number of candidate routes is denoted as N. R =|P f |=|P m |=|V|. In this application, the i-th leadership point is represented as... According to its route v i It can reach its destination. The neighbor's leader point is represented as in In a fixed image domain Find the K nearest neighbors of the dataset. In this application, K = 50 is set for all datasets. i The neighbor route is represented as in Depend on definition.

[0104] To quantify the clustering of V, we start with path consistency in the neighborhood, where path consistency is defined as v iand v j The directional correlation between. This metric only considers the direction of the route and is applicable in cases where the scale of motion is relatively small, such as the optical flow between video frames. However, in large deformations in image registration, although there is a high directional consistency in the large-scale motion of the route, there may be a large relative distance change before and after the motion, resulting in a low distance consistency, as shown in Figure 3 . A low distance consistency indicates severe elastic deformation, which may lead to image distortion. To quantify the elasticity of the deformation, the distance consistency is proposed as follows.

[0105] Only having high directional consistency does not necessarily guarantee high distance consistency. The arrows indicate the direction of the route, and the dashed lines represent the relative distances between the reference leading points and their neighbors. Although high directional consistency is achieved, indicating that the route directions are consistent, the relative distances between the reference leading points and their neighbors have changed significantly, resulting in a low distance consistency.

[0106] Given v i and v j , the relative distance in the fixed image is defined as Similarly, based on the destination and , the relative distance in the floating image is defined as Define v i and v j 's distance consistency as follows.

[0107]

[0108] where k dis ∈(0, +∞] controls the sensitivity of the distance consistency d(i, j). The closer k dis is to 0, the higher the penalty for d(i, j). The overall elasticity can be controlled by k dis . A larger k dis allows for a larger non-rigid deformation, while a smaller k dis will limit it to a rigid deformation.

[0109] The distance consistency limits the amount of relative distance change before and after the motion, but this metric does not strictly guarantee topology. As shown in Figure 4 , although the minimum relative distance change is achieved, the reference leading point goes beyond the solid line range after the motion, resulting in a low topological consistency. A makeshift solution is to use a smaller k dis to make the entire deformation field rigid, but this cannot compensate for non-rigid deformation. The Jacobian determinant (J) can represent the local topological preservation of the field, including dilation (J > 1), incompressibility (J = 1), contraction (0 < J < 1), and folding (J ≤ 0). In the field of finite element analysis, the Jacobian ratio (Jr This is typically used to calculate the maximum value J across all nodes. max With minimum value J min The ratio of J is used to test the shape of the element. max and J min If they are close, then J r Converging to 1 indicates that the element is similar to the initial identity element; if there is a negative J r This indicates that the element has been collapsed and failed the shape test. Although J... r It has proven effective in finite element analysis, but it cannot be directly applied to route consistency analysis because it measures the topology of the entire element rather than the route-level topology.

[0110] Inspired by Jacobian, the Jacobian route (J) was proposed. rr ) to quantify each route v i Its neighbor set The contribution of the topology. To achieve this, first define J. without (v i ) and J with (v i )as follows.

[0111]

[0112] Where J(p; TPS(V)) is the Jacobian determinant of the transformation of the TPS technique using a given set of routes V at point p. without (v i ) and J with (v i ) respectively represent the introduction of v i The topology before and after.

[0113] like Figure 4 As shown, high directional consistency and high distance consistency do not necessarily guarantee high topological consistency. The arrows indicate the direction of the route, and the dashed and solid lines represent the relative distances and topological structures between the reference leader point and its neighbors, respectively. Although high directional and distance consistency are achieved, the reference leader point moves beyond the range of the solid lines, resulting in lower topological consistency.

[0114] Given J without (v i ) and J with (v i The Jacobian route is defined as follows.

[0115]

[0116] Among them, J rr (vi ) represents v i Compared to its neighbors The topological changes are described. An Avalon-based voting method is proposed to efficiently compute J. rr (v i This will be introduced in the next subsection.

[0117] Based on J rr (v i A route-level topology was proposed, as shown below.

[0118]

[0119] Where, k top ∈(0,+∞] controls the sensitivity of the topology t(i). k top The closer J is to 0, rr The further away from 1, the lower t(i) becomes.

[0120] After calculating the topology of all routes in V, define v i and v j The topological consistency is as follows:

[0121] t(i,j) = t(i)·t(j)

[0122] Based on Avalon's vote:

[0123] J rr (v i The calculation depends on J without (v i ) and J with (v i If J without (v i ) is positive, indicating the introduction of v. i The previous topology was good, and J could be calculated. with (v i To easily isolate v i The contribution. Positive J with (v i ) indicates the introduction of v i The post-topology remains unchanged, indicating that v i It is a good topology route, leading to J rr (v i J is positive. Conversely, negative J is negative. with (v i ) indicates that the topology has been destroyed, and v i Marked as a bad topology route, resulting in J rr (v i ) is negative. However, when J without (v iWhen ) is negative (indicating the introduction of v) i (Previous topology was poor), determine v i The impact on the topology becomes challenging. Therefore, determining the route v i The key to topological quality lies in isolating its contributions, which involves selecting appropriate neighbor routes and calculating the contribution v. i Topological changes before and after.

[0124] This process is very similar to the board game Resistance: Avalon, a popular hidden-identity game involving two teams: good guys and bad guys. At the start of the game, players are randomly assigned to either the good guys or the bad guys team. In each round, a subset of players is chosen to perform a task. These players privately vote to decide whether the task succeeds or fails. If all players vote in favor, the task succeeds; otherwise, the task fails. In a straightforward Avalon game, the good guys always vote in favor, while the bad guys always vote out. To isolate the contribution of unconfirmed (TBC) players in a subset, the most efficient strategy is to introduce only one TBC player each time the vote is held, while the other players have already been identified as good guys. If the task fails, the TBC player is identified as a bad guy; otherwise, the TBC player is identified as a good guy.

[0125] Inspired by the Avalon game, this paper proposes an iterative method to identify good routes based on Avalon voting. Let V... Good V Evil and V TBC These are sets of routes: good, bad, and unconfirmed. These sets satisfy V. Good ∪V Evil ∪V TBC =V and In the first iteration, from Initially, all routes are unverified. For the x-th iteration, traverse... To identify each v i The role of this process involves three steps: selecting neighbor routes, calculating consistency changes, and determining route identities. A flowchart of this process is shown below. Figure 5 As shown.

[0126] Based on the voting process of Avalon, for the x-th iteration, traverse... Each route in the algorithm is identified by its contribution from isolated routes, including selecting neighboring routes, calculating consistency changes, and determining route identity. The identity of each route is determined in... Update in the middle, iteration continues until and There was no change.

[0127] Select a neighbor route:

[0128] According to the Avalon game strategy, by eliminating... All the bad guys and keep the good guys to isolate v i The contribution. First, choose a neighbor route that maintains the neighbors. The leader point remains one of the K nearest neighbors before and after the movement. Neighbor route v j The following two conditions must be met: 1. v j The starting point is in a fixed image domain Neighbors, 2, v j The destination is in the floating image domain The neighbors. (Introduction) As v i The set of K nearest neighbor routes, where the destination of the routes is in the floating image domain. The neighbors. Maintaining a subset of neighbors. The set of K nearest neighbor routes and Definition of intersection:

[0129]

[0130] in, yes and The two shots between them. and Determined based on each image domain f and m respectively.

[0131] Then, from Select a subset of the data as the anchor neighbor route, denoted as . Serves as an anchoring reference for quantifying changes in consistency. (Reserved) A good route, if possible, should be formed. Specifically, if The number of good routes in the data satisfies the minimum requirement for TPS interpolation, which is n+1. Only select... To build a good route Otherwise, merge the TBC set into the Good set to construct the collection.

[0132] Compute consistency changes: given Calculate J from the neighbor routes in the data. without (v i ) and J with (v i ) to quantify v i The consistency change relative to its neighboring routes. Equation J without (v i ) and J with (v i Updated as follows:

[0133]

[0134] Determine route identity: After completing the selection of neighbor routes and the calculation of consistency changes, determine v based on the results of these two steps. i The criteria for determining identity include the following three situations:

[0135] Good identity: The number of routes in J is at least n+1, representing the minimum number of routes required to satisfy TPS interpolation. Furthermore, J... without (v i ) and J with (v i All are positive, indicating the introduction of v i Post-consistency was well maintained. In this case, v i Marked as good, updated and Then quantize J rr (v i ).

[0136] Bad identity: If the number of routes in J is less than n+1, it means there are not enough routes for TPS interpolation. Alternatively, J without (v i ) is positive, but J with (v i A negative value indicates the introduction of v. i Post-consistency is broken. If any of the above conditions are met, v will... i Marked as bad, through update and Then punish J rr (v i ) = 0.

[0137] To be confirmed: The number of routes in J is at least n+1. However, J without (v i A negative value indicates the introduction of v. i The previous consistency was low. In this case, v i Its identity remains pending until the next iteration.

[0138] After iterating through all v i Afterwards, through and Update the set of good, bad, and undetermined routes. V is updated between two iterations. TBC When there is no change, that is All routes were marked as bad, accordingly. All values ​​are penalized to 0, ending the iteration.

[0139] Route decision and route inspection strategies

[0140] After integrating distance consistency, topological consistency, and orientation consistency, a specific route γ L The route consistency update is as follows:

[0141]

[0142] By utilizing the clustering of individuals, a consistency matrix Z can be calculated, which represents the route consistency between any two routes. Given a threshold k... thr A low-path consistency connection, i.e., Z(i,j). <k thr Removed components by setting Z(i,j) = 0. Then, collective clustering is applied as collective routes by searching for connected components that exceed the threshold Z. N C This represents the number of clusters in the collective route.

[0143] In this application, settings are provided for all datasets. in

[0144] Inspired by guiding animals to more suitable environments through appropriate migration routes, a route inspection strategy is proposed to determine the final route. This strategy measures the enhancement of local image similarity, enabling the identification and elimination of routes that negatively contribute. To achieve this, the convex region of each collective route is computed to obtain the region of interest (ROI). By solving the TPS matrix equation in parallel on a GPU, a GPU-based TPS technique was implemented, enabling efficient estimation of the displacement field for each ROI. The contribution of each collective route is defined as follows.

[0145]

[0146] Among them, c i It is Ω i The contribution of the i-th collective route. Sim(;) represents the similarity measure between two ROIs, which is defined as neighborhood correlation in this application.

[0147] By merging c i Clustering of positive collective routes yields the final collective routes, which can then drive large-scale deformation compensation across the entire image.

[0148]

[0149] Maintaining a collective approach to implementation:

[0150] Given the final collective route Route execution begins with interpolating the dense deformation field using TPS technology, known as flight. flight Driven by a collective path, it has the ability to effectively compensate for large-scale deformations.

[0151] After large-scale compensation, local small-scale deformations still appear in the image. These deformations can be easily estimated using traditional deformation models based on grayscale information. However, due to factors such as contrast agent and illumination variations, the grayscale of the image may change significantly. This can lead to image distortion when optimizing similarity metrics that rely solely on image grayscale information, thereby compromising the collective nature of the routes.

[0152] To address this problem, a route-stopping strategy is proposed, aiming to estimate a deformation field that preserves collectivity, termed diffusion. Diffusion helps maintain the leadership point on its original route while allowing the group to spread. This involves establishing a collective route. The drift distance is used as a penalty term in the objective function, as shown below.

[0153]

[0154] in, It is a normalized cross-correlation similarity, which is robust to changes in overall grayscale. The proposed route stops are defined as follows: To punish The drift distance further enhances robustness to local grayscale changes. Parameter k rs To balance similarity and penalty, a value of 0.01 is applied to all datasets. In this application, B-Spline transformation is used as... The deformation model is chosen because of its efficiency.

[0155] By optimizing the objective function Can use gradient descent technique to estimate To compensate for localized, small-scale deformations. Then, and Combinations lead to migration Compensate for large-scale and small-scale deformations in sequence.

[0156] Cascade migration with collective inheritance

[0157] Traditional methods typically converge to local optima after initial transformations and non-rigid deformations. In contrast, this method possesses the ability to further optimize through cascaded migrations. Notably, each migration enhances image similarity, aiding in the identification of candidate routes. This, in turn, enables the generation of the next migration to drive the next collective movement. To facilitate multiple migrations, a cascaded migration framework incorporating a route merging strategy is introduced.

[0158] Similar to the multiple migrations animals undertake to reach their destination, the cascading migration framework involves continuously deforming a floating image m through multiple migrations. Let k cas Let y be the maximum number of migrations, and let y be the number of migrations. The yth time m is defined as Cascade migration from m 0 =m starts when the group route cannot be determined The migration converges over time. The convergence condition ensures that increasing k... cas This can improve registration accuracy without significantly increasing computation time. In this application, k is set for all datasets. cas =3.

[0159] In cascading migrations, ensuring the collectivity of the route is crucial. During migration... Subsequently, most of the large-scale deformations were effectively compensated, leaving only a small amount of scattered deformation distributed in small areas of the image. In the subsequent migration... Flights generated from these small areas This may cause image drift due to the previous migration. The route in the middle is not in Consideration is needed.

[0160] To address this issue, a route merging strategy is proposed, which involves merging the current collective routes. merged into the previous collective route Update from here:

[0161]

[0162] By employing a route merging strategy, the collectivity of routes in cascade migration is ensured, improving the robustness and accuracy of registration. Once the cascade migration converges, the floating image m is deformed... Align with the fixed image f at its height.

[0163] In summary, the large-scale deformation compensation method provided in this application treats salient points in an image as leaders within a group, similar to the alpha animal in a migrating flock, playing a crucial role in guiding the entire registration process. To estimate the clustering manifold, large-scale deformation registration is divided into three parts according to the animal's migration pattern: route decision based on clustering quantization, route execution with clustering preservation, and cascade migration with clustering inheritance. These three parts are integrated into the Clustering Cascade Migration (CCM) framework, effectively compensating for large-scale image deformation. This method significantly improves registration accuracy and robustness when compensating for large-scale deformation. It is a general approach, not limited to specific feature descriptors or deformation models, and can be easily extended to other image registration tasks. Because this method utilizes the concept of clustering motion to provide a new approach to the large-scale deformation problem, it will bring more scalability to the field of image registration.

[0164] See Figure 6 This application embodiment can also provide a large-scale deformation compensation device, such as... Figure 6 As shown, the apparatus for performing the above-described large-scale deformation compensation method may include:

[0165] The candidate route identification unit 601 is used to extract features from a fixed image and a floating image to obtain feature extraction results, and to identify several candidate routes by using a feature matching method and a parameter ratio test threshold in combination with the feature extraction results. The candidate routes are the potential correspondence between feature points in the fixed image and feature points in the floating image.

[0166] The consistency matrix acquisition unit 602 is used to quantify the consistency of several candidate routes using the route consistency quantization module to obtain a consistency matrix; the route consistency quantization module is used to perform direction consistency quantization, distance consistency quantization and topology consistency quantization on several candidate routes.

[0167] Clustering route determination unit 603 is used to determine several clustering routes from the consistency matrix using a clustering clustering algorithm;

[0168] The final collective route determination unit 604 is used to determine a final collective route from a plurality of said clustered routes using a route checking strategy. The final collective route is used to drive large-scale deformation compensation of the image. The route checking strategy is used to enhance the measurement of the similarity of local images in order to exclude negatively contributing routes.

[0169] Flight field acquisition unit 605 is used to obtain the flight field of the guiding group by thin plate spline interpolation of the final collective route, wherein the flight field is a large-scale dense deformation field;

[0170] The diffusion field acquisition unit 606 is used to obtain the diffusion field by combining the route dwelling strategy with the flight field estimation. The diffusion field is a small-scale dense deformation field.

[0171] The migration field acquisition unit 607 is used to combine the flight field and the diffusion field into a deformation field to obtain a migration field; the migration field is used to promote the deformation of the floating image, and to perform large-scale deformation compensation and small-scale deformation compensation on the floating image during the deformation process.

[0172] The cascaded migration unit 608 is used to continuously deform the floating image through multiple migrations using a cascaded migration framework and the migration field until the migration converges to obtain the target deformed image.

[0173] This application embodiment can also provide a large-scale deformation compensation device, the device including a processor and a memory:

[0174] The memory is used to store program code and transmit the program code to the processor;

[0175] The processor is used to execute the steps of the large-scale deformation compensation method described above according to the instructions in the program code.

[0176] like Figure 7 As shown in the embodiment of this application, a large-scale deformation compensation device may include: a processor 10, a memory 11, a communication interface 12, and a communication bus 13. The processor 10, memory 11, and communication interface 12 all communicate with each other through the communication bus 13.

[0177] In this embodiment, the processor 10 may be a central processing unit (CPU), an application-specific integrated circuit, a digital signal processor, a field-programmable gate array, or other programmable logic devices.

[0178] The processor 10 can call the program stored in the memory 11. Specifically, the processor 10 can execute the operations in the embodiments of the large-scale deformation compensation method.

[0179] The memory 11 is used to store one or more programs. The programs may include program code, which includes computer operation instructions. In this embodiment, the memory 11 stores at least a program for implementing the following functions:

[0180] Feature extraction is performed on fixed and floating images to obtain feature extraction results. Then, a feature matching method and a parameter ratio test threshold are used in combination with the feature extraction results to identify several candidate routes. The candidate routes are the potential correspondence between feature points in the fixed image and feature points in the floating image.

[0181] A route consistency quantization module is used to quantify the consistency of several candidate routes to obtain a consistency matrix; the route consistency quantization module is used to perform direction consistency quantization, distance consistency quantization and topology consistency quantization on several candidate routes.

[0182] A clustering algorithm is used to determine several clustering routes from the consensus matrix;

[0183] A route checking strategy is employed to determine a final collective route from several clustered routes, the final collective route being used to drive large-scale deformation compensation of the image; the route checking strategy is also used to enhance the measurement of the similarity of local images in order to exclude routes that contribute negatively.

[0184] The flight field guiding the group is obtained by thin-plate spline interpolation of the final collective route, and the flight field is a large-scale dense deformation field;

[0185] The diffusion field is obtained by combining the route dwell strategy with the flight field estimation. The diffusion field is a small-scale dense deformation field.

[0186] The flight field and the diffusion field are combined to form a deformation field to obtain a migration field; the migration field is used to promote the deformation of the floating image, and to perform large-scale deformation compensation and small-scale deformation compensation on the floating image during the deformation process.

[0187] The floating image is continuously deformed through multiple migrations using a cascaded migration framework and the migration field until the migration converges, thus obtaining the target deformed image.

[0188] In one possible implementation, the memory 11 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function (such as file creation or data read / write). The data storage area may store data created during use, such as initialization data.

[0189] In addition, memory 11 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device or other volatile solid-state storage device.

[0190] Communication interface 12 can be an interface for the communication module, used to connect with other devices or systems.

[0191] Of course, it should be noted that, Figure 7 The structure shown does not constitute a limitation on the large-scale deformation compensation device in the embodiments of this application. In practical applications, the large-scale deformation compensation device may include devices larger than […]. Figure 7 More or fewer components as shown, or combinations of certain components.

[0192] This application embodiment may also provide a computer-readable storage medium for storing program code for executing the steps of the above-described large-scale deformation compensation method.

[0193] It should be noted that in this application, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0194] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of this application.

[0195] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for system or system embodiments, since they are basically similar to method embodiments, the description is relatively simple, and relevant parts can be referred to the descriptions in the method embodiments. The systems and system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0196] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.

Claims

1. A method for large-scale deformation compensation, characterized in that, include: Feature extraction is performed on fixed and floating images to obtain feature extraction results. Then, a feature matching method and a parameter ratio test threshold are used in combination with the feature extraction results to identify several candidate routes. The candidate routes are the potential correspondence between feature points in the fixed image and feature points in the floating image. A route consistency quantization module is used to quantify the consistency of several candidate routes to obtain a consistency matrix; the route consistency quantization module is used to perform direction consistency quantization, distance consistency quantization and topology consistency quantization on several candidate routes. A clustering algorithm is used to determine several clustering routes from the consensus matrix; A route checking strategy is employed to determine a final collective route from several clustered routes, the final collective route being used to drive large-scale deformation compensation of the image; the route checking strategy is also used to enhance the measurement of the similarity of local images in order to exclude routes that contribute negatively. The flight field guiding the group is obtained by thin-plate spline interpolation of the final collective route, and the flight field is a large-scale dense deformation field; The diffusion field is obtained by combining the route dwell strategy with the flight field estimation. The diffusion field is a small-scale dense deformation field. The flight field and the diffusion field are combined to form a deformation field to obtain a migration field; the migration field is used to promote the deformation of the floating image, and to perform large-scale deformation compensation and small-scale deformation compensation on the floating image during the deformation process. The floating image is continuously deformed through multiple migrations using a cascaded migration framework and the migration field until the migration converges, thus obtaining the target deformed image.

2. The large-scale deformation compensation method according to claim 1, characterized in that, The most lenient ratio test is used to determine all feature points with bijective matching as candidate routes.

3. The large-scale deformation compensation method according to claim 1, characterized in that, The distance consistency is represented by the following formula: In the formula: k dis ∈(0, +∞) is used to control the sensitivity of distance consistency d(i,j), d m d represents the relative distance in the floating image. f Represents relative distances in a fixed image; A voting method based on Avalon is used for topological consistency quantification, whereby topological consistency is represented by the following formula: t(i,j) = t(i)·t(j) In the formula: t(i) represents the route v i The route-level topology, t(j) represents route v j Route-level topology, route v j Indicates route v i The neighbor's route.

4. The large-scale deformation compensation method according to claim 3, characterized in that, t(i) is expressed by the following formula: In the formula: k top ∈(0, +∞) is used to control the sensitivity of the topology t(i), J rr (v i ) represents v i Compared to its neighbors Topological changes; The J rr (v i It can be expressed by the following formula: In the formula: J without (v i ) and J with (v i ) respectively represent the introduction of v i The topology before and after.

5. The large-scale deformation compensation method according to claim 1, characterized in that, The final collective route is represented by the following formula: In the formula: c i Represents Ω i The contribution of the i-th collective route, Ω i This represents the i-th region of interest.

6. The large-scale deformation compensation method according to claim 1, characterized in that, The route stay strategy collective route The drift distance, as a penalty term in the objective function, is expressed by the following formula: In the formula: L represents the normalized cross-correlation similarity. rs This indicates a stop on the route, defined as follows: k rs The parameter represents the trade-off between similarity and penalty. Indicates airfield, Indicates the diffusion field.

7. The large-scale deformation compensation method according to claim 1, characterized in that, Before the cascaded migration converges, the current collective route is merged into the previous collective route. In, it is represented by the following formula: In the formula: Indicates the current collective route. This indicates the route taken by the previous group.

8. A large-scale deformation compensation device, characterized in that, The apparatus for performing the large-scale deformation compensation method according to any one of claims 1-7 includes: The candidate route identification unit is used to extract features from fixed images and floating images to obtain feature extraction results, and to identify several candidate routes by using a feature matching method and a parameter ratio test threshold in combination with the feature extraction results. The candidate routes are the potential correspondence between feature points in the fixed image and feature points in the floating image. The consistency matrix acquisition unit is used to quantify the consistency of several candidate routes using the route consistency quantization module to obtain a consistency matrix; the route consistency quantization module is used to perform direction consistency quantization, distance consistency quantization and topology consistency quantization on several candidate routes. A clustering route determination unit is used to determine several clustering routes from the consistency matrix using a clustering clustering algorithm; The final collective route determination unit is used to determine a final collective route from several said clustered routes using a route checking strategy. The final collective route is used to drive large-scale deformation compensation of the image. The route checking strategy is used to enhance the measurement of the similarity of local images in order to exclude negatively contributing routes. The flight field acquisition unit is used to obtain the flight field of the guiding group by thin plate spline interpolation of the final collective route. The flight field is a large-scale dense deformation field. The diffusion field acquisition unit is used to obtain the diffusion field by combining the route dwelling strategy with the flight field estimation. The diffusion field is a small-scale dense deformation field. The migration field acquisition unit is used to combine the flight field and the diffusion field into a deformation field to obtain a migration field; the migration field is used to promote the deformation of the floating image, and to perform large-scale deformation compensation and small-scale deformation compensation on the floating image during the deformation process. A cascaded migration unit is used to continuously deform the floating image through multiple migrations using a cascaded migration framework and the migration field until the migration converges and the target deformed image is obtained.

9. A large-scale deformation compensation device, characterized in that, The device includes a processor and a memory: The memory is used to store program code and transmit the program code to the processor; The processor is used to execute the large-scale deformation compensation method according to any one of claims 1-7 according to the instructions in the program code.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store program code for executing the large-scale deformation compensation method according to any one of claims 1-7.

Images