Image registration method for composite schlieren technology

Through the image registration method of global-local structure consistency of feature points, the unreliability of velocity and density field measurement and the difficulty of accurate optical path adjustment in composite pattern technology are solved, and the automatic alignment and density of four camera images are realized. , accurate calculation of velocity field.

CN118967767AInactive Publication Date: 2024-11-15INST OF HIGH SPEED AERODYNAMICS OF CHINA AERODYNAMICS RES & DEV CENT
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
CN202411441848.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-10-16
Publication Date
2024-11-15
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

The existing composite pattern technology is difficult to accurately measure the speed and density fields simultaneously in high-speed flow fields, resulting in unreliable measurement results and difficult to achieve accurate optical path adjustment, resulting in misalignment of objects and images and large calculation errors.

Method used

Using the image registration method based on global-local structure consistency of feature points, feature points are extracted through SIFT operator, initial correspondence relationship is constructed, false matching is removed, spatial transformation matrix is ​​calculated, image alignment is achieved using bilinear interpolation, and automatic matching is done. Quasi-four camera images.

Benefits of technology

Automatic image registration is realized without the need to accurately adjust the relative relationship between the light spot and the photosensitive element, reducing the error in the calculation density and velocity field, and meeting the measurement needs of high-speed flow fields.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118967767A_ABST
    Figure CN118967767A_ABST
Patent Text Reader

Abstract

The invention belongs to the technical field of aerospace and experimental aerodynamics, and particularly relates to an image registration method for a composite schlieren technology. According to the technical scheme, the image registration method for the composite schlieren technology comprises the following steps that a calibration plate image and an experiment image are shot, specifically, before an experiment, a calibration plate is placed in a light path, and the calibration plate image is shot; after the calibration plate is removed, the positions of the light path and the camera are kept unchanged, and an experiment image is shot; an image registration method based on global-local structure consistency of feature points is adopted and comprises the steps of feature extraction, feature rough matching, mismatching removal, spatial transformation matrix estimation and image transformation. The relative relation between the light spot and the photosensitive element does not need to be accurately adjusted, and it only needs to ensure that the object image is completely imaged on the target surface. Through calibration and a specific algorithm, the four images are automatically registered so as to meet the requirements of calculating a density field and a velocity field.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention belongs to the field of aerospace and experimental aerodynamics technology, and in particular relates to an image registration method for composite schlieren technology. Background Art

[0002] Measuring Reynolds stress in flow fields has always been a difficult problem in the field of experimental fluid mechanics, because from the formula point of view, in order to measure Reynolds stress, it is necessary to measure the velocity and density field in the flow at the same time, which is very difficult. Limited by technical difficulties, scholars usually use the method of measuring density and velocity separately, but in high-speed flow fields, velocity and density are highly coupled, and the results of separate measurements are often unreliable.

[0003] In recent years, the High Speed ​​Aerodynamics Research Institute of the China Aerodynamics Research and Development Center has developed a system that can measure density and velocity simultaneously, namely a compressible Reynolds stress measurement system based on composite schlieren technology. The system uses a tetrahedron to replace the blade of the traditional schlieren, and uses four camera target surfaces to receive four beams of light after passing through the tetrahedron as input, and uses intensity shearing, wavefront reconstruction and other technologies to reconstruct the density field. For velocity measurement, the system uses a pulsed laser light source, combined with a cross-frame camera, and applies the optical flow method to calculate the velocity. Due to the needs of technologies such as intensity shearing and wavefront reconstruction, the same pixel position on the four target surfaces is required to represent the same point in space.

[0004] For the existing composite schlieren technology, when calculating the density field, it is necessary to align the object images on the four camera target surfaces behind the tetrahedron so that the positions of the object images on the target surfaces are consistent, such as Figure 2 As shown, otherwise, it is impossible to calculate reliable density and velocity fields. Therefore, this requires the operator to adjust the positions of the circular light spots and square photosensitive elements on the four cameras when adjusting the optical path, so as to ensure that the images on the four cameras can overlap. However, precise adjustment is basically impossible to achieve. In actual operation, the target surface of the camera will not remain fixed in position as in the ideal state, and the four images are always misaligned, such as Figure 3 Therefore, when calculating the density and velocity fields, there will be large errors or even calculation errors. Summary of the invention

[0005] In order to solve the above problems existing in the prior art, an object of the present invention is to provide an image registration method for composite schlieren technology.

[0006] The technical solution adopted by the present invention is: An image registration method for composite schlieren technology comprises the following steps: Take the calibration plate image and experimental image: Before the experiment, place the calibration plate in the light path and take the calibration plate image; then remove the calibration plate, ensure that the light path and the camera are in the same position, and take the experimental image; The image registration method based on the global-local structural consistency of feature points is adopted: S1: Feature extraction: Use SIFT operator to perform feature detection on the calibration plate images collected by multiple cameras; S2: Feature rough matching: construct an initial set of correspondence relationships based on the similarity between feature point descriptors; S3: false match removal; S4: Estimate the spatial transformation matrix: Use the preserved correspondence to make the two sets of feature points correspond one to one, consider the influence of bilateral noise, and use the total least squares idea to calculate the spatial transformation matrix; S5: Image transformation: Use step S4 to obtain the spatial transformation matrix between the two images. Take the image captured by the first camera lens as the reference, and use the bilinear interpolation method to transform the images captured by the remaining cameras according to the transformation matrix, so that all camera images are aligned with the image of the first camera.

[0007] The present invention uses a calibration plate with obvious feature images to meet the requirements of SIFT feature extraction. An image registration method based on the global-local structural consistency of feature points is used to achieve automatic image registration. The present invention does not need to accurately adjust the relative relationship between the light spot and the photosensitive element, but only needs to ensure that the object image is completely imaged on the target surface. Through calibration and specific algorithms, the four images are automatically registered to meet the requirements of calculating the density field and the velocity field.

[0008] As a preferred embodiment of the present invention, step S1 includes the following specific steps: S11: Build a multi-resolution image pyramid for each image to be detected, search images in all scale spaces, and identify potential interest points that are invariant to scale and selection through Gaussian derivative functions; S12: Positioning feature points based on the stability of candidate points; S13: Assign directions to key points according to the gradient direction of the local structure and describe each feature point.

[0009] As a preferred solution of the present invention, in step S2, the correspondence set is composed of the feature point pairs that match each other in the feature point sets of the two images. Composition, among which, is the feature point in the first feature point set, is a feature point in the second feature point set.

[0010] As a preferred embodiment of the present invention, step S3 includes the following specific steps: S31: Remove wrong matches in the rough matching set through local structural similarity and construct a cost function; S32: Introduce Hausdorff's neighborhood structure consistency constraint to obtain a new cost function; S33: Derive the closed-form solution of the cost function.

[0011] As a preferred solution of the present invention, in step S31, assuming that there is only a rigid transformation between the two images, the distance between the corresponding point pairs is maintained; assuming is an unknown set of interior points, then the optimal interior point set is (1); Cost Function The structure is: (2); in, is the Euclidean distance, represents the number of elements in a set, (C, ) represents the cost parameter of the cost function S, parameter Controls the balance between quadratic and compensatory terms, express The coordinates of the point, express The coordinates of the points; for the preserved local structure, the above cost function is rewritten as: (3); In the formula, express The neighborhood point set is composed of the K nearest neighbor points of each point under the Euclidean distance. No. Correspondence The matching correctness of Represents point x j In x i a field Inside, Represents point y j In y i a field Internal; used in the cost function To normalize the contribution of each element in the neighborhood; the distance between point pairs based on the local structure consistency constraint can be quantified as: (4); (5); The correspondence relationship and the binary vector association, its elements Indicates Correspondence The matching correctness of indicates a correct match, otherwise it indicates an incorrect match; then the cost function is simplified to: (6); in, (7); It means to calculate the number of elements in a collection. Represents two neighborhood sets and The total number of elements.

[0012] As a preferred solution of the present invention, in step S32, after introducing the Hausdorff neighborhood structure consistency constraint, the new cost function is: (8); In the formula, represents the new cost parameter after the introduction of Hausdorff, Represents a collection of two fields, Represents the quantized value defined by Hausdorff distance.

[0013] As a preferred solution of the present invention, in step S33, a multi-scale neighborhood representation is introduced. Algorithm: Assume For feature points In the Euclidean distance The cost function of the multi-scale neighborhood is: (9); Minimizing formula (9) can remove the wrong matches and get the correct correspondence. The relevant items can be combined to obtain: (10); in, (11); By setting a reasonable threshold, the corresponding relationships with less similarity can be removed and the corresponding relationships with higher accuracy can be retained.

[0014] As a preferred solution of the present invention, step S4 is specifically: Assume that the observation data matrix and The following linear equation is satisfied: (12); represents the optimal solution of equation (12); TLS introduces two different perturbation matrices and To eliminate the contradiction in formula (12); minimize two different perturbation matrices and The energy of , formula (12) is rewritten as: (13); (14); in, represents the optimal solution of the equation after the perturbation matrix is ​​introduced, The dimension is The real matrix of Indicated by , The transformation matrix obtained by least squares of and Y is, represents the square of the Frobenius norm, is an augmented matrix composed of two perturbation matrices; the homogeneous form of the above formula is: (15); (16); in, and are the unit matrix and zero moment respectively; the spatial transformation matrix is ​​obtained by derivation using singular value decomposition.

[0015] The beneficial effects of the present invention are: The present invention uses a calibration plate with obvious feature images to meet the requirements of SIFT feature extraction. An image registration method based on the global-local structural consistency of feature points is used to achieve automatic image registration. The present invention does not need to accurately adjust the relative relationship between the light spot and the photosensitive element, but only needs to ensure that the object image is completely imaged on the target surface. Through calibration and specific algorithms, the four images are automatically registered to meet the requirements of calculating the density field and the velocity field. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 is a flow chart of the image registration method of the present invention; Figure 2 This is the relationship diagram between the light spot and the target surface under ideal conditions; Figure 3 It is the relationship diagram between the light spot and the target surface when the object is not aligned; Figure 4 It is a high-definition calibration plate image. DETAILED DESCRIPTION

[0017] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0018] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present invention. It should be noted that the embodiments of the present invention and the features in the embodiments can be combined with each other without conflict.

[0019] like Figure 1 As shown, the present invention is mainly used for the calibration and post-processing of the composite schlieren system, so that the operator can realize the automatic alignment of the object image without having to accurately adjust the relative position of the light spot and the photosensitive element behind the four-sided pyramid, thereby realizing the accurate calculation of density and speed. In order to realize this technology, it is necessary to carry out the following aspects: First, before the experiment, place the calibration plate in the light path and take a picture of the calibration plate. Then, remove the calibration plate, make sure the light path and the camera are in place, and take pictures of the experiment. The first camera takes a picture of the calibration plate. Figure 4 shown.

[0020] The image registration method based on the global-local structural consistency of feature points is adopted, as follows: S1: Feature extraction: The SIFT operator is used to perform feature detection on the calibration plate images collected by multiple cameras.

[0021] S11: Build a multi-resolution image pyramid for each image to be detected, search images in all scale spaces, and identify potential interest points that are invariant to scale and selection through Gaussian derivative functions; S12: Positioning feature points based on the stability of candidate points; S13: Assign directions to key points according to the gradient direction of the local structure and describe each feature point.

[0022] S2: Feature rough matching: Construct an initial correspondence set based on the similarity between feature point descriptors. The correspondence set consists of the feature point pairs that match each other in the feature point sets of the two images. Composition, among which, is the feature point in the first feature point set, is a feature point in the second feature point set.

[0023] S3: False match removal: S31: Remove the wrong matches in the coarse matching set by local structural similarity. Construct a cost function. Assuming that there is only a rigid transformation between the two images, the distance between the corresponding point pairs can be maintained. Assume is an unknown set of interior points, then the optimal interior point set is (1); Cost Function The structure is: (2); in, is the Euclidean distance, represents the number of elements in a set, (C, ) represents the cost parameter of the cost function S, parameter Controls the balance between quadratic and compensatory terms, express The coordinates of the point, express The coordinates of the points; for the preserved local structure, the above cost function is rewritten as: (3); In the formula, express The neighborhood point set is composed of the K nearest neighbor points of each point under the Euclidean distance. No. Correspondence The matching correctness of Represents point x j In x i a field Inside, Represents point y j In y i a field Internal; used in the cost function To normalize the contribution of each element in the neighborhood; the distance between point pairs based on the local structure consistency constraint can be quantified as: (4); (5); The correspondence relationship and the binary vector association, its elements Indicates Correspondence The matching correctness of indicates a correct match, otherwise it indicates an incorrect match; then the cost function is simplified to: (6); in, (7); It means to calculate the number of elements in a collection. Represents two neighborhood sets and The total number of elements.

[0024] S32: After introducing Hausdorff's neighborhood structure consistency constraint, the new cost function is: (8); In the formula, represents the new cost parameter after the introduction of Hausdorff, Represents a collection of two fields, Represents the quantized value defined by Hausdorff distance.

[0025] S33: Derive the closed-form solution of the cost function. Since the K nearest neighbors are searched under the Euclidean distance, the different distribution of the neighborhood set elements on the image and the different proportions of outliers will lead to different numbers of optimal neighbors, so the optimal value of K is not fixed. In order to solve this problem, a multi-scale neighborhood representation is introduced. Algorithm: Assume For feature points In the Euclidean distance The cost function of the multi-scale neighborhood is: (9); Minimizing formula (9) can remove the wrong matches and get the correct correspondence. The relevant items can be combined to obtain: (10); in, (11); By setting a reasonable threshold, the corresponding relationships with less similarity can be removed and the corresponding relationships with higher accuracy can be retained.

[0026] S4: Estimate the spatial transformation matrix: Use the preserved correspondence to make the two sets of feature points correspond one to one, consider the influence of bilateral noise, and use the total least squares idea to calculate the spatial transformation matrix. Assume that the observed data matrix and The following linear equation is satisfied: (12); represents the optimal solution of equation (12); TLS introduces two different perturbation matrices and To eliminate the contradiction in formula (12); minimize two different perturbation matrices and The energy of , formula (12) is rewritten as: (13); (14); in, represents the optimal solution of the equation after the perturbation matrix is ​​introduced, The dimension is The real matrix of Indicated by , The transformation matrix obtained by least squares of and Y is, represents the square of the Frobenius norm, is an augmented matrix composed of two perturbation matrices; the homogeneous form of the above formula is: (15); (16); in, and are the unit matrix and zero moment respectively; the spatial transformation matrix is ​​obtained by derivation using singular value decomposition.

[0027] S5: Image transformation: use step S4 to obtain the spatial transformation matrix between the two images. Take the image captured by the first camera lens as the reference, and use the bilinear interpolation method to transform the images captured by the remaining cameras according to the transformation matrix, so that all camera images are aligned with the image of the first camera.

[0028] The present invention adopts a calibration plate with obvious feature images to meet the requirements of SIFT feature extraction. An image registration method based on the global-local structural consistency of feature points is used to realize automatic image registration.

[0029] The present invention does not need to precisely adjust the relative relationship between the light spot and the photosensitive element, but only needs to ensure that the object image is completely imaged on the target surface. Through calibration and specific algorithms, the four images are automatically registered to meet the requirements of calculating the density field and the velocity field.

[0030] The present invention has been verified by experiments. After roughly adjusting the relative position of the light spot and the photosensitive element, the density field and velocity field calculated by the registration algorithm of the present invention are close to the theoretical and simulated values, and the error meets the requirements.

[0031] The present invention is not limited to the above-mentioned optional implementation modes. Anyone can derive other various forms of products under the inspiration of the present invention. However, no matter what changes are made in the shape or structure, all technical solutions that fall within the scope defined by the claims of the present invention fall within the protection scope of the present invention.

Claims

1. An image registration method for composite schlieren technology, Features: The following steps are involved: Take the calibration plate image and experimental image: Before the experiment, place the calibration plate in the light path and take the calibration plate image; then remove the calibration plate, ensure that the light path and the camera are in the same position, and take the experimental image; The image registration method based on the global-local structural consistency of feature points is adopted: S1: Feature extraction: Use SIFT operator to perform feature detection on the calibration plate images collected by multiple cameras; S2: Feature rough matching: construct an initial set of correspondence relationships based on the similarity between feature point descriptors; S3: false match removal; S4: Estimate the spatial transformation matrix: Use the preserved correspondence to make the two sets of feature points correspond one to one, consider the influence of bilateral noise, and use the total least squares idea to calculate the spatial transformation matrix; S5: Image transformation: Use step S4 to obtain the spatial transformation matrix between the two images. Take the image captured by the first camera lens as the reference, and use the bilinear interpolation method to transform the images captured by the remaining cameras according to the transformation matrix, so that all camera images are aligned with the image of the first camera.

2. The image registration method for composite schlieren technology according to claim 1, Features: Step S1 includes the following specific steps: S11: Build a multi-resolution image pyramid for each image to be detected, search images in all scale spaces, and identify potential interest points that are invariant to scale and selection through Gaussian derivative functions; S12: Positioning feature points based on the stability of candidate points; S13: Assign directions to key points according to the gradient direction of the local structure and describe each feature point.

3. The image registration method for composite schlieren technology according to claim 1, Features: In step S2, the correspondence set is composed of the feature point pairs that match each other in the feature point sets of the two images. Composition, among which, is the feature point in the first feature point set, is a feature point in the second feature point set.

4. The image registration method for composite schlieren technology according to claim 3, Features: Step S3 includes the following specific steps: S31: Remove wrong matches in the rough matching set through local structural similarity and construct a cost function; S32: Introduce Hausdorff's neighborhood structure consistency constraint to obtain a new cost function; S33: Derive the closed-form solution of the cost function.

5. An image registration method for composite schlieren technology according to claim 4, Features: In step S31, assuming that there is only a rigid transformation between the two images, the distance between the corresponding point pairs is maintained; is an unknown set of interior points, then the optimal interior point set is (1); Cost Function The structure is: (2); in, is the Euclidean distance, represents the number of elements in a set, (C, ) represents the cost parameter of the cost function S, parameter Controls the balance between quadratic and compensatory terms, express The coordinates of the point, express The coordinates of the points; for the preserved local structure, the above cost function is rewritten as: (3); In the formula, express The neighborhood point set is composed of the K nearest neighbor points of each point under the Euclidean distance. No. Correspondence The matching correctness of Represents point x j In x i a field Inside, Represents point y j In y i a field Internal; used in the cost function To normalize the contribution of each element in the neighborhood; the distance between point pairs based on the local structure consistency constraint can be quantified as: (4); (5); The correspondence relationship and the binary vector association, its elements Indicates Correspondence The matching correctness of indicates a correct match, otherwise it indicates an incorrect match; then the cost function is simplified to: (6); in, (7); It means to calculate the number of elements in a collection. Represents two neighborhood sets and The total number of elements.

6. An image registration method for composite schlieren technology according to claim 4, Features: In step S32, after introducing Hausdorff's neighborhood structure consistency constraint, the new cost function is: (8); In the formula, represents the new cost parameter after the introduction of Hausdorff, Represents a collection of two fields, Represents the quantized value defined by Hausdorff distance.

7. An image registration method for composite schlieren technology according to claim 4, Features: In step S33, a multi-scale neighborhood representation is introduced Algorithm: Assume For feature points In the Euclidean distance The cost function of the multi-scale neighborhood is: (9); Minimizing formula (9) can remove the wrong matches and get the correct correspondence. The relevant items can be combined to obtain: (10); in, (11); By setting a reasonable threshold, the corresponding relationships with less similarity can be removed and the corresponding relationships with higher accuracy can be retained.

8. The image registration method for composite schlieren technology according to claim 4, Features: Step S4 is specifically as follows: Assume that the observation data matrix and The following linear equation is satisfied: (12); represents the optimal solution of equation (12); TLS introduces two different perturbation matrices and To eliminate the contradiction in formula (12); minimize two different perturbation matrices and The energy of , formula (12) is rewritten as: (13); (14); in, represents the optimal solution of the equation after the perturbation matrix is ​​introduced, The dimension is A real matrix of Indicated by , The transformation matrix obtained by least squares of and Y is, represents the square of the Frobenius norm, is an augmented matrix composed of two perturbation matrices; the homogeneous form of the above formula is: (15); (16); in, and are the unit matrix and zero moment respectively; the spatial transformation matrix is ​​obtained by derivation using singular value decomposition.

Citation Information

Patent Citations

  • Star image distortion detection and estimation method based on star map matching

    CN103440659A

  • Rapid vehicle-mounted laser point cloud automatic registration method in complex urban environment

    CN117152215A

  • Target-free camera-radar calibration method based on line segments

    CN118501829A

  • Image registration method of multi-camera imaging system

    CN118762059A

  • High-precision dynamic real-time 360-degree omnidirectional point cloud acquisition method based on fringe projection

    US20240037765A1