Image registration method and device based on image neighborhood deviation correction

Abstract

The invention discloses an image registration method and device based on image neighborhood deviation correction, and belongs to the technical field of image registration, and the method comprises the following steps: obtaining an initial perspective transformation matrix; wherein the initial perspective transformation matrix is obtained by performing preliminary registration on a reference image and a to-be-registered image; constructing a brightness area in the reference image to obtain a reference display image; acquiring a shot image according to the initial perspective transformation matrix and the reference display image, and calculating a real shot brightness center of each brightness area in the shot image; obtaining a point pair set of the theoretical center coordinate and the real shot brightness center coordinate of each brightness area, and calculating a secondary perspective mapping matrix through the point pair set; iteratively updating the brightness area to obtain a plurality of secondary perspective mapping matrixes; and fusing the plurality of secondary perspective mapping matrixes to obtain a final matrix. According to the invention, the registration precision can be improved from a pixel level to a sub-pixel level, and the high-precision application requirement is met.

Description

Technical Field

[0001] This invention belongs to the field of image registration technology, and particularly relates to an image registration method and apparatus based on image neighborhood deviation correction. Background Technology

[0002] Image registration is a core technology in computer vision, widely used in industrial inspection, XR image synthesis, medical imaging, and remote sensing. Common registration algorithms include SIFT, SURF, or ORB algorithms.

[0003] For example, patent CN116503451A discloses a method and system for target registration and positioning of inspection images from fixed cameras in substations. The method includes: S100. Setting inspection points, acquiring a first round of inspection images, using the first round of inspection images as a reference image to be registered, and marking the coordinate points of the target area in the reference image; S200. Acquiring inspection images and registering them to the same coordinate system as the reference image; S300. Calculating the transformation matrix based on the marked coordinate points of the target area in the reference image and inputting the marked target area coordinate points into a perspective transformation matrix; S400. Obtaining the target area image from the registered inspection image using the coordinate point information and the perspective transformation matrix. Specifically, in S200, the method for registering the inspection image to the same coordinate system as the reference image is as follows: first, a SIFT feature descriptor is used to extract feature points from the two images, and then a BF matcher is used to match the feature points of the two images.

[0004] However, current conventional registration algorithms such as SIFT, SURF, or ORB have the following drawbacks when registering reference and target images:

[0005] Limited accuracy: Traditional algorithms are limited by the accuracy of feature extraction and matching, making it difficult to achieve sub-pixel level registration accuracy, which cannot meet the needs of high-precision applications such as industrial micron-level detection or film and television-level visual synthesis.

[0006] Environmental sensitivity: It is sensitive to image quality, lighting changes, occlusion and noise, and has poor registration stability in complex scenes.

[0007] Insufficient real-time performance: The iterative optimization process is computationally complex and difficult to run efficiently in real-time systems, and it lacks continuous tracking and adaptive re-registration mechanisms.

[0008] Therefore, there is an urgent need to develop an image registration method and apparatus based on image neighborhood deviation correction to solve the problems in the existing technology. Summary of the Invention

[0009] The purpose of this invention is to provide an image registration method and apparatus based on image neighborhood deviation correction, which can achieve registration accuracy at the sub-pixel level, thereby solving the problem of poor registration accuracy of traditional algorithms mentioned in the background art.

[0010] To solve the above-mentioned technical problems, the specific technical solution of the present invention is as follows:

[0011] An image registration method based on image neighborhood bias correction includes the following steps:

[0012] Obtain the initial perspective transformation matrix; wherein, the initial perspective transformation matrix is ​​obtained by performing preliminary registration on the reference image and the image to be registered;

[0013] Construct a brightness region in the reference image to obtain the reference display image;

[0014] The captured image is obtained based on the initial perspective transformation matrix and the reference display image, and the actual brightness center of each brightness region in the captured image is calculated.

[0015] Obtain a set of point pairs between the theoretical center coordinates and the actual brightness center coordinates of each brightness region, and calculate the secondary perspective mapping matrix using the point pair set;

[0016] Iteratively update the brightness region to obtain several secondary perspective mapping matrices; fuse these secondary perspective mapping matrices to obtain the final matrix.

[0017] Furthermore, the brightness of the brightness area gradually decreases from the center outwards.

[0018] Furthermore, the maximum error of the initial registration does not exceed D pixels, and the brightness area is a circular area with a radius of D×2.

[0019] Furthermore, the coordinates of the brightness center in the actual image are calculated using a convolution formula, and the summation range covers the entire brightness area.

[0020] Furthermore, the second-order perspective mapping matrix satisfies the following equation:

[0021] ;

[0022] in, The second perspective mapping matrix is ​​a 3×3 matrix; the theoretical center coordinates are... ; Actual brightness center coordinates .

[0023] Furthermore, the fusion of several quadratic perspective mapping matrices is calculated using the following formula:

[0024] ;

[0025] in, This is the final matrix; For the first The weights for each iteration are set based on the magnitude of the deviation; the smaller the deviation, the greater the weight. n and wj are parameters. This represents the number of iterations.

[0026] Furthermore, it also includes:

[0027] Calculate the theoretical center coordinates and the corresponding actual brightness center coordinates for each brightness region to obtain the deviation value, which is used to evaluate the registration quality.

[0028] Furthermore, it also includes:

[0029] If the deviation exceeds the preset threshold, the iterative registration process is restarted to ensure that the system adapts to changes in the environment.

[0030] A computer device includes a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method.

[0031] A computer program product includes a computer program that, when executed by a processor, implements the steps of the method.

[0032] This application also has the following beneficial technical effects:

[0033] (1) By iteratively updating the brightness region to obtain several secondary perspective mapping matrices, and then fusing several secondary perspective mapping matrices to obtain the final matrix, the registration accuracy is improved from the conventional pixel level to the sub-pixel level, thus meeting the requirements of high-precision applications.

[0034] (2) Through the circular brightness area, the gradient brightness design and the multi-position distribution of the brightness area, it effectively resists local occlusion, lighting changes and noise interference.

[0035] (3) By using the theoretical center coordinates and the actual brightness center coordinates in the registration process, the deviation value can be calculated, supporting real-time tracking and dynamic re-registration. It is suitable for dynamic scenarios such as industrial online detection and XR real-time synthesis. The entire process does not require manual intervention, reducing deployment and maintenance costs. Attached Figure Description

[0036] Figure 1 This is a schematic diagram of the overall process of the present invention;

[0037] Figure 2 This is a schematic diagram of the initial registration in Example 1 of the present invention;

[0038] Figure 3 This is a schematic diagram of the white brightness area in Example 1 of the present invention;

[0039] Figure 4 This is a schematic diagram of the neighborhood deviation correction in Example 1 of the present invention;

[0040] Figure 5 This is a schematic diagram of the preliminary registration in Example 2 of the present invention;

[0041] Figure 6 This is a schematic diagram of the neighborhood deviation correction in Case 2 of the present invention. Detailed Implementation

[0042] To better understand the purpose, structure, and function of this invention, the invention will be described in further detail below with reference to the accompanying drawings.

[0043] An image registration method based on image neighborhood bias correction, such as Figure 1 As shown, it includes the following steps:

[0044] S1: Obtain the initial perspective transformation matrix; wherein, the initial perspective transformation matrix is ​​obtained by performing preliminary registration on the reference image and the image to be registered;

[0045] S2: Construct a brightness region in the reference image to obtain the reference display image;

[0046] S3: Obtain the captured image based on the initial perspective transformation matrix and the reference display image, and calculate the actual brightness center of each brightness region in the captured image;

[0047] S4: Obtain the set of point pairs between the theoretical center coordinates and the actual brightness center coordinates of each brightness region, and calculate the secondary perspective mapping matrix using the point pair set.

[0048] S5: Iteratively update the brightness region to obtain several secondary perspective mapping matrices; fuse several secondary perspective mapping matrices to obtain the final matrix.

[0049] In step S1, the initial perspective transformation matrix is ​​obtained. Initial registration involves using conventional registration algorithms such as SIFT, SURF, or ORB. The maximum error in initial registration should not exceed D pixels to ensure basic alignment. The value of D ranges from 3 to 30. The image after initial registration is shown below. Figure 2 and Figure 5 As shown.

[0050] Specifically, registration using conventional registration algorithms is a prior art. For example, the SIFT algorithm can be found in Lwe, DG "Distinctive image features from scale-invariant keys." International Journal of Cmputer Visin, 2004. The SURF algorithm can be found in Bay, H., et al. "SURF: SpeededUp Rbust Features." Cmputer Visin and Image Understanding, 2008. The ORB algorithm can be found in Rublee, E., et al. "RB: An efficient alternative to SIFT and SURF." International Conference in Cmputer Visin, 2011. Therefore, this application will not elaborate further.

[0051] In this embodiment, the brightness area in S2 is a circular area.

[0052] Step S2 involves constructing a brightness region in the reference image to obtain a reference display image, including the following steps:

[0053] Select no fewer than four display positions in the reference image, and display a circular area with a radius of D×2 in each display position to obtain the reference display image.

[0054] The color of the circular area is optimized based on the imaging system. In this embodiment, for example... Figure 3 As shown, the circular area is white.

[0055] The brightness of the brightness area gradually decreases from the center outwards to enhance the robustness of brightness center detection. The display position is typically located around the perimeter of the image to avoid obscuring the subject.

[0056] The actual brightness center in S3 is the energy center of gravity.

[0057] After obtaining the captured image in S3, the actual brightness center coordinates of each brightness region in the captured image are calculated using a convolution formula.

[0058] The convolution formula is as follows:

[0059] ;

[0060] in, The coordinates of the brightness center; For point The brightness value; the summation range covers the entire brightness area.

[0061] This embodiment also includes the following steps:

[0062] The theoretical center coordinates and the corresponding actual image center coordinates of each brightness region are calculated to obtain the deviation value, which is used to evaluate the registration quality. The deviation value between the brightness center coordinates and the theoretical center coordinates is the registration error.

[0063] In step S4, a set of point pairs is obtained between the theoretical center coordinates and the actual brightness center coordinates of each brightness region to obtain the theoretical center coordinates. and the center coordinates of the actual brightness The set of point pairs constituted, where, .

[0064] The second-order perspective mapping matrix in S4 satisfies the following equation:

[0065] ;

[0066] in, The matrix is ​​a 3×3 homography matrix, which is a quadratic perspective mapping matrix. It is solved by least squares method or RANSAC algorithm, and the specific solution process is existing technology, which will not be described in this application.

[0067] In step S5, the brightness region is iteratively updated to obtain several secondary perspective mapping matrices, which are obtained by repeating steps S2-S4.

[0068] The S5 incorporates several quadratic perspective mapping matrices, calculated using the following formula:

[0069] ;

[0070] in, This is the final matrix; For the first The weights for each iteration are set based on the magnitude of the deviation; the smaller the deviation, the greater the weight. n and wj are parameters; generally, n=5 and wj=1 can be chosen to achieve good results. Optimization and adjustment should be made based on actual measurements. This represents the number of iterations.

[0071] In this embodiment, iteration stops when the registration deviation reaches the sub-pixel level. For example, the error is < 0.5 pixels.

[0072] Finally, based on the final matrix, the image to be registered is transformed to obtain and output the subpixel-level registration result.

[0073] In this embodiment, the sub-pixel level registration result is as follows: Figure 4 and Figure 6 As shown.

[0074] It also includes the following steps:

[0075] If the deviation exceeds a preset threshold, the iterative registration process is restarted to ensure the system adapts to environmental changes. The iterative registration process refers to step S5. In this embodiment, the preset threshold is... Pixels, set as needed.

[0076] like Figures 2 to 4 As the first example of this application, the actual measured data after applying the procedure of this application are as follows:

[0077] ~15:59:52.560: Start position alignment optimization: Current offset: 1.97

[0078] ~15:59:52.568: Position alignment optimization: 0 Target: 0.500 Fine-tune current offset: 1.968 New offset: 1.377

[0079] ~15:59:52.830: Position alignment optimization: 1 Target: 0.500 Fine-tune current offset: 1.778 New offset: 1.039

[0080] ~15:59:53.103: Position alignment optimization: 2 Target: 0.500 Fine-tune current offset: 1.585 New offset: 0.958

[0081] ~15:59:53.497: Position alignment optimization: 3 Target: 0.500 Fine-tuning current offset: 1.333 New offset: 0.694

[0082] ~15:59:53.754: Position alignment optimization: 4 Target: 0.500 Fine-tuning current offset: 1.177 New offset: 0.644

[0083] ~15:59:54.226: Position alignment optimization complete. Current offset: 0.494. Target: 0.500.

[0084] like Figure 5 and Figure 6 This is Case Two of this application. After applying the procedure of this application, the actual measured data is as follows:

[0085] 16:00:24.337: Start position alignment optimization: Current offset: 12.58

[0086] 16:00:24.340: Position alignment optimization: 0 Target: 0.500 Fine-tuning current offset: 12.584 New offset: 10.8

[0087] 16:00:25.133: Position alignment optimization: 6 Target: 0.500 Fine-tuning current offset: 11.154 New offset: 8.71

[0088] 16:00:25.399: Position alignment optimization: 7 Target: 0.500 Fine-tuning current offset: 10.697 New offset: 3.72

[0089] 16:00:25.667: Position alignment optimization: 8 Target: 0.500 Fine-tuning current offset: 8.574 New offset: 2.458

[0090] 16:00:26.100: Position alignment optimization: 9 Target: 0.500 Fine-tuning current offset: 5.723 New offset: 2.794

[0091] 16:00:26.367: Position alignment optimization: 10 Target: 0.500 Fine-tuning current offset: 4.789 New offset: 2.84

[0092] 16:00:26.766: Position alignment optimization: 11 Target: 0.500 Fine-tuning current offset: 4.133 New offset: 3.09

[0093] 16:00:27.001: Position alignment optimization: 12 Target: 0.500 Fine-tuning current offset: 3.873 New offset: 2.33

[0094] 16:00:27.433: Position alignment optimization: 13 Target: 0.500 Fine-tuning current offset: 3.294 New offset: 1.79

[0095] 16:00:27.667: Position alignment optimization: 14 Target: 0.500 Fine-tuning current offset: 2.835 New offset: 1.02

[0096] 16:00:28.076: Position alignment optimization: 15 Target: 0.500 Fine-tuning current offset: 2.226 New offset: 1.05

[0097] 16:00:28.324: Position alignment optimization: 16 Target: 0.500 Fine-tuning current offset: 1.897 New offset: 0.79

[0098] 16:00:28.700: Position alignment optimization: 17 Target: 0.500 Fine-tuning current offset: 1.463 New offset: 0.70

[0099] 16:00:28.965: Position alignment optimization: 18 Target: 0.500 Fine-tuning current offset: 1.213 New offset: 0.67

[0100] 16:00:29.394: Position alignment optimization complete. Current offset: 0.495. Target: 0.500.

[0101] In summary, this application has the following technical effects:

[0102] (1) Subpixel-level registration accuracy: Through iterative deviation correction and multi-estimation fusion, the registration accuracy is improved from pixel level to subpixel level to meet the requirements of high-precision applications.

[0103] (2) Strong anti-interference capability: The gradual brightness design of the circular area and the distribution of multiple positions effectively resist local occlusion, lighting changes and noise interference.

[0104] (3) Low environmental requirements: No complex hardware or strict environmental conditions are required, and it is suitable for ordinary imaging systems.

[0105] (4) Real-time and adaptive: Supports real-time tracking and dynamic re-registration, suitable for dynamic scenarios such as industrial online inspection and XR real-time synthesis.

[0106] (5) High degree of automation: The entire process does not require manual intervention, which reduces deployment and maintenance costs.

[0107] A system comprising:

[0108] Standard imaging equipment used to acquire captured images;

[0109] A display, used to show a reference image or a reference display image;

[0110] A computer device includes a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method.

[0111] In this embodiment, the standard imaging device can be an industrial camera or a regular webcam, and the computer device can be a GPU workstation.

[0112] In this embodiment, the method and system are mainly applied in the field of live video processing technology, used for XR image synthesis, and to achieve the fusion of virtual and real scenes.

[0113] Optionally, the method can also be used for: industrial fine target detection, such as PCB board alignment and part measurement; medical image analysis, such as multimodal image registration; remote sensing and map stitching, etc.

[0114] A computer program product includes a computer program that, when executed by a processor, implements the steps of the method.

[0115] The final matrix is ​​applied to transform the image to be registered, and the subpixel-level registration result is obtained and output.

[0116] It is understood that the present invention has been described through some embodiments, and those skilled in the art will recognize that various changes or equivalent substitutions can be made to these features and embodiments without departing from the spirit and scope of the invention. Furthermore, under the teachings of the present invention, these features and embodiments can be modified to adapt to specific situations and materials without departing from the spirit and scope of the invention. Therefore, the present invention is not limited to the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of this application are within the protection scope of the present invention.

[0117] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. An image registration method based on image neighborhood bias correction, characterized by, The method comprises the following steps: obtaining an initial perspective transformation matrix, wherein the initial perspective transformation matrix is obtained by performing preliminary registration on a reference image and a to-be-registered image; constructing a brightness region in the reference image to obtain a reference display image; obtaining a photographed image according to the initial perspective transformation matrix and the reference display image, and calculating a real brightness center of each brightness region in the photographed image; obtaining a point pair set of a theoretical center coordinate and a real brightness center coordinate of each brightness region, and calculating a secondary perspective mapping matrix through the point pair set; iteratively updating the brightness region to obtain a plurality of secondary perspective mapping matrices, and fusing the plurality of secondary perspective mapping matrices to obtain a final matrix.

2. The image registration method based on image neighborhood bias correction according to claim 1, characterized in that, The brightness of the brightness region gradually decreases from the center to the periphery.

3. The image registration method based on image neighborhood bias correction according to claim 2, characterized in that, The maximum error of the preliminary registration is not more than D pixels, and the brightness region is a circular region with a radius of Dx2.

4. The image registration method based on image neighborhood bias correction according to claim 3, characterized in that, The real brightness center is calculated by a convolution formula, and the summation range covers the entire brightness region.

5. The image registration method based on image neighborhood bias correction according to claim 4, characterized in that, The secondary perspective mapping matrix satisfies the following formula: ； wherein, is a secondary perspective mapping matrix, is a 3x3 matrix; theoretical circle center coordinate ; real shot brightness center coordinate .

6. The image registration method based on image neighborhood bias correction according to claim 5, wherein, The plurality of secondary perspective mapping matrices are fused by the following formula: ； wherein, is the final matrix; is the weight of the first iteration, which is set according to the deviation value, the smaller the deviation, the greater the weight; is the weight of the first iteration, which is set according to the deviation value, the smaller the deviation, the greater the weight; is the number of iterations.

7. The image registration method based on image neighborhood bias correction according to any one of claims 1-6, characterized in that, The method further comprises: calculating a theoretical center coordinate and a corresponding real brightness center coordinate of each brightness region to obtain a deviation value, which is used to evaluate the registration quality.

8. The image registration method based on image neighborhood bias correction of claim 7, wherein, The method further comprises: If the deviation value exceeds a preset threshold, the iterative registration process is restarted to ensure that the system is adaptive to environmental changes.

9. A computer apparatus comprising a memory, a processor, and a computer program stored on the memory, wherein the computer program, when executed by the processor, causes the processor to perform the method of any one of claims 1 to 8. The processor executes the computer program to implement the steps of the method according to any one of claims 1-8.

10. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method according to any one of claims 1-8.

Images