An image alignment method based on tensor multi-view clustering

Abstract

The application relates to an image alignment method based on tensor multi-view clustering and belongs to the technical field of machine vision and remote sensing, and is used for solving the problems of illumination change, shielding, pixel damage and attitude / view angle misalignment existing in real image acquisition. The application uses high-dimensional multi-view subspace characteristics of images as priori, extracts low-dimensional category information through tensor product, represents a multi-view similar matrix in a tensor form, constructs a tensor low-rank regularization term to depict a low-dimensional subspace of image categories, simultaneously introduces a tensor sparse regularization term to constrain alignment errors, and jointly forms a tensor multi-view clustering alignment model. A generalized Gauss-Newton algorithm is used to linearize a nonlinear domain transformation, and model variables are solved through an alternating direction multiplier method. Experimental results show that the method can effectively retain high-dimensional structures of images, and the alignment accuracy and detail restoration effect are better than those of mainstream algorithms such as RASL and OSLECT, and the robustness and practicability are stronger.

Description

Technical Field

[0001] This invention relates to the fields of machine vision and remote sensing technology, specifically to an image alignment method based on tensor multi-view clustering. Background Technology

[0002] With the rapid development of new sensors and computer science, image processing technology has been widely applied in fields such as face recognition, Earth observation, and medical imaging. However, problems such as illumination variations, occlusion, pixel corruption, and image misalignment exist during image acquisition. Processing this unconstrained real-world visual data poses a significant challenge to existing methods. Image processing problems can be broadly categorized based on their specific nature, such as image denoising, image deblurring, image enhancement and restoration, and image alignment. These problems have significant practical applications in our lives, work, and research. In many cases, image alignment is necessary before further processing. Therefore, image alignment is a meaningful area of ​​research.

[0003] On the one hand, hardware improvements can address image misalignment, but this leads to higher image acquisition costs. Currently, the most common approach is to combine imaging technology with digital tools, leveraging mathematical modeling to overcome the limitations of imaging hardware. For example, traditional restoration methods based on filtering, sparse representation, and low-rank matrix regularization ignore the high-dimensional structure of the data, reducing their ability to represent the underlying image and causing distortion of information within the image. In recent years, artificial intelligence methods, represented by deep learning, have gained increasing popularity in image restoration due to their big data-driven advantages; however, their underlying mathematical mechanisms remain unclear, and the large parameter scale in the models reduces their stability and transferability, affecting the algorithm's generalization ability and the accuracy of image alignment restoration. However, these methods ignore the inherent high-dimensional linear structure of images. Driven by high-performance computing capabilities, it has become possible to analyze, implement, and apply large-scale structured tensor data, effectively processing the data and extracting more useful information. Therefore, we combine the clustering advantages of tensors in high-dimensional space to learn useful category information from the input image, achieving accurate image alignment.

[0004] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of the present invention, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0005] This invention provides an image alignment method based on tensor multi-view clustering, which solves the problem that existing image alignment methods ignore the high-dimensional tensor structure of the image, do not utilize multi-view category priors, and are prone to information distortion due to traditional methods. Furthermore, the deep learning method has unclear mechanisms and poor generalization, making it difficult to solve the problem of accurate image alignment in real-world scenarios affected by lighting, occlusion, and pose shift.

[0006] Other features and advantages of the invention will become apparent from the following detailed description, or may be learned in part by practice of the invention.

[0007] According to a first aspect of the present invention, an image alignment method based on tensor multi-view clustering is provided, the method comprising: Step 1: Construct a tensor-based image alignment model, which models the unaligned observed image data as an aligned low-rank tensor superimposed with a sparse error tensor after domain transformation. This transforms the image alignment problem into a mathematical problem of recovering the low-rank tensor and the sparse error tensor from the observed data. Step 2: Based on the model established in Step 1, a tensor-based multi-view clustering image alignment model is further constructed. This involves using low-rank tensor representations to characterize the high-dimensional structure and category information of the image. The aligned low-rank tensor is decomposed into the tensor product of feature tensors and associated tensors. Tensor kernel norm constraints are applied to the associated tensors to enhance the characterization of the low-dimensional subspace. Simultaneously, the tensor of sparse error is incorporated. l By applying the 1-norm constraint, a joint tensor low-rank representation and sparse regularization objective function are established, and the generalized Gauss-Newton linear approximation is applied to the domain transformation to obtain the linearized optimization model. Step 3: Solve the linearized model obtained in Step 2 using the alternating direction multiplier method. That is, introduce auxiliary variables to construct an augmented Lagrangian function, and alternately update the low-rank tensor, correlation tensor, sparse error tensor, transformation parameters and Lagrangian multipliers until convergence, and finally output the aligned image.

[0008] In some exemplary embodiments, the tensor-based image alignment model described in step 1 specifically refers to: For the original sample data We obtain the result by performing a domain transformation.

[0009] in, , Operator symbol Indicates the transformation symbol, This represents the aligned image. These represent the length, width, and number of sample images, respectively. Aligned image Rewritten as:

[0010] The image alignment problem is transformed into a mathematical expression, namely, unaligned and noisy observation data. Recovering the low-rank tensor and sparse tensors .

[0011] In some exemplary embodiments, the image alignment model based on tensor multi-view clustering described in step 2 is specifically as follows:

[0012] in, Represents the tensor nuclear norm. Tensor l 1-norm, For sparse regularization parameters, Represents the characteristic tensor. Represents the correlation tensor. Represents the tensor product.

[0013] In some exemplary embodiments, the generalized Gaussian-Newton linear approximation of the domain transformation to obtain the linearized optimized model is specifically as follows: When the domain transformation parameter Change value When I was very young, Performing a first-order Taylor expansion, we obtain A linear approximation of , namely

[0014] in, Indicates the first Images with respect to transformation parameters Jacobian matrix, express The standard base, let

[0015] in, This indicates that a tensor is synthesized along the third dimension; The linearized optimization model is as follows: .

[0016] In some exemplary embodiments, step 3 specifically includes: Introducing auxiliary variables and make The equivalent form of the linearized optimization model can be obtained as follows:

[0017] Solving the above equation directly is difficult. By using the alternating direction multiplier method, the constraint problem in the above equation can be expressed as the following augmented Lagrangian function:

[0018] in, It is a Lagrange multiplier. It is the penalty term parameter; in the framework of the alternating direction multiplier method, each variable in the solution is solved alternately by fixing other variables; (I) Optimization Optimization: With other variables fixed, The objective function is:

[0019] in, ; (II) Optimization Optimization: With other variables fixed, The objective function is:

[0020] This is a least squares problem with nuclear norm regularity, which can be solved using readily available tensor singular value threshold shrinkage operators; (III) Optimization Optimization: With other variables fixed, The objective function is:

[0021] The solution can be obtained using readily available tensor threshold shrinkage operators. ; (IV) Optimization Optimization: With other variables fixed, The objective function is:

[0022] Solving using the proximal gradient method ,as follows:

[0023] (V) Update Lagrange multipliers :

[0024] Until Output aligned image .

[0025] According to a second aspect of the present invention, a storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the image alignment method based on tensor multi-view clustering described in the first aspect.

[0026] According to a third aspect of the present invention, a computer program product is provided, on which a computer program is stored, wherein when the computer program is executed by a processor, it implements the image alignment method based on tensor multi-view clustering described in the first aspect above.

[0027] According to a fourth aspect of the present invention, an electronic device is provided, comprising: Processor; and Memory for storing the executable instructions of the processor; The processor is configured to implement the tensor multi-view clustering-based image alignment method described in the first aspect above by executing the executable instructions.

[0028] The image alignment method based on tensor multiview clustering provided by embodiments of the present invention utilizes the multiview low-dimensional structure of images in tensor space. Specifically, the present invention utilizes the multiview subspace characteristics of images in high-dimensional space and uses these characteristics as prior information to guide the low-rank representation of images. First, the category information of data in the low-dimensional subspace is obtained using tensor product. By constructing appropriate low-dimensional features or dictionaries, we perform multiview clustering on the image features. In this process, we perform tensor quantization representation on the similarity matrices of different views, introduce a regularization term of low-rank tensor to enhance the characterization of the low-dimensional subspaces of different categories in the image, and derive a low-rank tensor representation model. At the same time, since there is an error between the input image and the aligned image, we introduce a tensor sparse regularization term to constrain this error. By combining tensor low-rank representation and sparse regularization, we obtain an image alignment algorithm based on tensor multiview clustering (TMC). Finally, the proposed alignment algorithm is solved using the alternating direction multiplier method. Experimental results show that the method proposed in this invention achieves excellent image alignment performance. The beneficial effects are as follows: 1. Preserve high-dimensional structure and enhance expressive power: Tensor modeling fully preserves the high-dimensional structure of the image, which is closer to the characteristics of real data than traditional matrix methods and reduces information distortion.

[0029] 2. Combining low-rank and sparse constraints enhances robustness. Tensor low-rank regularization: accurately characterizes the low-dimensional subspace of image categories and improves structural representation.

[0030] Tensor sparsity regularization: effectively constrains alignment errors, noise and anomalies, significantly improving anti-interference capabilities.

[0031] 3. Multi-view clustering guidance for more accurate alignment: By leveraging tensor product to extract category information and using multi-view clustering as a priori guidance for low-rank representation, the alignment accuracy is superior to conventional methods.

[0032] 4. The solution is efficient and stable. The Alternating Direction Multiplier Method (ADMM) is used for decomposition optimization, combined with the generalized Gauss-Newton linear approximation, which results in fast model convergence and computational stability.

[0033] 5. Balancing alignment and reconstruction for wider applicability: It not only completes image alignment but also restores detailed information, making it suitable for various scenarios such as face recognition, multi-temporal remote sensing, and medical imaging.

[0034] Experimental verification shows that this invention outperforms mainstream comparative methods, such as RASL and OSLECT. l 1+ADMM, l p +ADMM, this method offers the best performance in detail restoration and alignment.

[0035] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit the invention. Attached Figure Description

[0036] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention. It is obvious that the drawings described below are merely some embodiments of the invention, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0037] Figure 1 This is a flowchart of the method described in this application.

[0038] Figure 2 It is the image alignment result on the publicly available dataset used for testing. Detailed Implementation

[0039] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, they are provided so that the invention will be more comprehensive and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0040] Furthermore, the accompanying drawings are merely illustrative of the invention and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0041] To address the shortcomings and deficiencies of existing technologies, this example embodiment provides an image alignment and reconstruction method based on tensor multi-view clustering. This method can be used to solve problems such as illumination variations, occlusion, pixel corruption, and image misalignment during image acquisition, including issues like misaligned image acquisition postures in face image recognition and inconsistent acquisition perspectives at different times in multi-temporal remote sensing images. Specifically, it may include the following steps: Step 1: Construct a tensor-based image alignment model, which models the unaligned observed image data as an aligned low-rank tensor superimposed with a sparse error tensor after domain transformation. This transforms the image alignment problem into a mathematical problem of recovering the low-rank tensor and the sparse error tensor from the observed data. Step 2: Based on the model established in Step 1, a tensor-based multi-view clustering image alignment model is further constructed. This involves using low-rank tensor representations to characterize the high-dimensional structure and category information of the image. The aligned low-rank tensor is decomposed into the tensor product of feature tensors and associated tensors. Tensor kernel norm constraints are applied to the associated tensors to enhance the characterization of the low-dimensional subspace. Simultaneously, the tensor of sparse error is incorporated. l By applying the 1-norm constraint, a joint tensor low-rank representation and sparse regularization objective function are established, and the generalized Gauss-Newton linear approximation is applied to the domain transformation to obtain the linearized optimization model. Step 3: Solve the linearized model obtained in Step 2 using the alternating direction multiplier method. That is, introduce auxiliary variables to construct an augmented Lagrangian function, and alternately update the low-rank tensor, correlation tensor, sparse error tensor, transformation parameters and Lagrangian multipliers until convergence, and finally output the aligned image.

[0042] The steps in this exemplary embodiment will now be described in more detail with reference to the accompanying drawings and examples.

[0043] The numerical data (number '3') used in this embodiment comes from a publicly available dataset. Comparison methods include Robust Alignment by Sparse and Low-rank Decomposition (RASL), Optimum Subspace Learning and Error Correction for Tensors (OSLECT), and methods based on... l Alternating Direction Multiplier Method with Norm 1 (ADMM) l 1-norm, l 1+ADMM) method, based on l p Alternating Direction Multiplier Method with Lp-norm l p +ADMM) method. The flowchart of this alignment method is as follows: Figure 1 As shown.

[0044] Step 1: Construct a tensor-based image alignment model: When the position or pose of the target object relative to the camera changes slightly, images of the same object or scene may also show significant differences, and a set of images of the same sampled object is highly likely to be misaligned. We can model misalignment as a domain transformation, such as affine transformation, similarity transformation, translation transformation, etc. We use the original sample data... We obtain the result by performing a domain transformation on the top. (1) in, Operator symbol Indicates the transformation symbol, This represents the aligned image. Let represent the length, width, and number of sample images, respectively. It is worth noting that equation (1) also indicates that the domain transformation is applied to each image. Therefore, (1) can be rewritten as: (2) The image alignment problem is transformed into a mathematical expression for unaligned and noisy observation data. Recovering the low-rank tensor and sparse tensors .

[0045] Step 2: Construct an image alignment model based on tensor multi-view clustering To characterize the inherent high-dimensional structure of images and their corresponding category information, we introduce, for example, low-rank models based on tensor representations, which can represent lower-dimensional structural information, i.e. Decomposed into characteristic tensors and correlation tensor The product of these terms, and the relationship between them described by tensor product, are as follows: (3) in, This represents the tensor product. It can be obtained by considering the associated tensors. Applying low-rank constraints yields a more efficient low-dimensional subspace characterization model, as follows: (4) in, This represents the nuclear norm of the tensor. Combining (2), (4), and... Due to the sparsity of the data, we derive an image alignment model based on tensor multi-view clustering, as follows: (5) in, Tensor l 1-norm, For sparse regularization parameters. In (5) domain transformation The transformation could be linear or nonlinear. If it's nonlinear, directly solving problem (5) is difficult. Therefore, we apply the generalized Gauss-Newton algorithm to solve (5). The main idea of ​​the generalized Gauss-Newton algorithm is to make a linear approximation of the components of the target nonlinear model. A widely used linearization method is when the domain transformation parameters Change value When I was very young, Performing a first-order Taylor expansion, we obtain A linear approximation of , namely (6) in, Indicates the first Images with respect to transformation parameters Jacobian matrix, express The standard base. Let (7) in, This indicates that a tensor is synthesized along the third dimension. Therefore, (5) can be rewritten as: (8) Step 3: Solve the alignment problem using the alternating direction multiplier method. To facilitate the solution, we introduce auxiliary variables. and make The equivalent form of equation (8) can be obtained as follows: (9) Directly solving (9) is difficult. By using the method of alternating direction multipliers, the constraint problem in formula (9) can be expressed as the following augmented Lagrangian function: (10) in, It is a Lagrange multiplier. It is the penalty term parameter. Under the alternating direction multiplier method framework, each variable in (7) is solved alternately by fixing other variables.

[0046] (I) Optimization Optimization: With other variables fixed, The objective function is: (11) in, .

[0047] (II) Optimization Optimization: With other variables fixed, The objective function is: (12) This is a least squares problem with nuclear norm regularity, which can be solved using the readily available tensor singular value threshold shrinkage operator.

[0048] (III) Optimization Optimization: With other variables fixed, The objective function is: (13) We can solve this using readily available tensor threshold shrinkage operators. .

[0049] (IV) Optimization Optimization: With other variables fixed, The objective function is: (14) Solving using the proximal gradient method ,as follows: (15) (V) Update Lagrange multipliers : (16) Algorithm 1 summarizes the above optimization process.

[0050]

[0051] This application embodiment uses the method of this application to align the image of the number '3'. The method of this application can reconstruct detailed information during the alignment process.

[0052] To qualitatively evaluate the inpainting effect, this invention selected four benchmark image inpainting algorithms: RASL method, OSLECT method, and... l 1+ADMM method l p +ADMM is used as a comparison method, and the restoration results of the aligned image are compared with the results of the present invention using these comparison methods.

[0053] Visual comparison results as follows Figure 2 As shown, it can be observed that all methods can align the spatial structure in the image well. From the comparison of details, the technique of the present invention obviously has the best performance in restoring the detailed features of the image. This proves that the proposed technique can not only achieve image alignment, but also has a good image reconstruction capability.

[0054] Furthermore, the above figures are merely illustrative of the processes included in the method according to exemplary embodiments of the present invention, and are not intended to be limiting. It is readily understood that the processes shown in the above figures do not indicate or limit the temporal order of these processes. Additionally, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0055] Other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention herein. This application is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein. The specification and embodiments are to be considered exemplary only, and the true scope and spirit of the invention are indicated by the claims.

[0056] It should be understood that the present invention is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is defined only by the appended claims.

Claims

1. An image alignment method based on tensor multi-view clustering, characterized in that, The method includes: Step 1: Construct a tensor-based image alignment model, which models the unaligned observed image data as an aligned low-rank tensor superimposed with a sparse error tensor after domain transformation. This transforms the image alignment problem into a mathematical problem of recovering the low-rank tensor and the sparse error tensor from the observed data. Step 2: Based on the model established in Step 1, a tensor-based multi-view clustering image alignment model is further constructed. This involves using low-rank tensor representations to characterize the high-dimensional structure and category information of the image. The aligned low-rank tensor is decomposed into the tensor product of feature tensors and associated tensors. Tensor kernel norm constraints are applied to the associated tensors to enhance the characterization of the low-dimensional subspace. Simultaneously, the tensor of sparse error is incorporated. l By applying the 1-norm constraint, a joint tensor low-rank representation and sparse regularization objective function are established, and the generalized Gauss-Newton linear approximation is applied to the domain transformation to obtain the linearized optimization model. Step 3: Solve the linearized model obtained in Step 2 using the alternating direction multiplier method. That is, introduce auxiliary variables to construct an augmented Lagrangian function, and alternately update the low-rank tensor, correlation tensor, sparse error tensor, transformation parameters and Lagrangian multipliers until convergence, and finally output the aligned image.

2. The method according to claim 1, characterized in that, The tensor-based image alignment model described in step 1 is specifically as follows: For the original sample data We obtain the result by performing a domain transformation. in, , Operator symbol Indicates the transformation symbol, This represents the aligned image. These represent the length, width, and number of sample images, respectively. Aligned image Rewritten as: The image alignment problem is transformed into a mathematical expression, namely, unaligned and noisy observation data. Recovering the low-rank tensor and sparse tensors .

3. The method according to claim 2, characterized in that, The image alignment model based on tensor multi-view clustering described in step 2 is specifically as follows: in, Represents the tensor nuclear norm. Tensor l 1-norm, For sparse regularization parameters, Represents the characteristic tensor. Represents the correlation tensor. Represents the tensor product.

4. The method according to claim 3, characterized in that, The generalized Gaussian-Newton linear approximation of the domain transformation yields the linearized optimized model, specifically: When the domain transformation parameter Change value When I was very young, Performing a first-order Taylor expansion, we obtain A linear approximation of , namely in, Indicates the first Images with respect to transformation parameters Jacobian matrix, express The standard base, let in, This indicates that a tensor is synthesized along the third dimension; The linearized optimization model is as follows: 。 5. The method according to claim 4, characterized in that, Step 3 specifically involves: Introducing auxiliary variables and make The equivalent form of the linearized optimization model can be obtained as follows: Solving the above equation directly is difficult. By using the alternating direction multiplier method, the constraint problem in the above equation can be expressed as the following augmented Lagrangian function: in, It is a Lagrange multiplier. It is the penalty term parameter; in the framework of the alternating direction multiplier method, each variable in the solution is solved alternately by fixing other variables; (I) Optimization Optimization: With other variables fixed, The objective function is: in, ; (II) Optimization Optimization: With other variables fixed, The objective function is: This is a least squares problem with nuclear norm regularity, which can be solved using readily available tensor singular value threshold shrinkage operators; (III) Optimization Optimization: With other variables fixed, The objective function is: The solution can be obtained using readily available tensor threshold shrinkage operators. ; (IV) Optimization Optimization: With other variables fixed, The objective function is: Solving using the proximal gradient method ,as follows: (V) Update Lagrange multipliers : Until Output aligned image .

6. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the image alignment method based on tensor multi-view clustering as described in any one of claims 1 to 5.

7. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the image alignment method based on tensor multi-view clustering as described in any one of claims 1 to 5.

8. An electronic device, characterized in that, include: processor; as well as Memory for storing the executable instructions of the processor; The processor is configured to execute the image alignment method based on tensor multi-view clustering as described in any one of claims 1 to 5 by executing the executable instructions.

Images