Multi-image registration fusion method for low-calculation-degree edge device

Abstract

The invention discloses a multi-image registration fusion method for a low-calculation-degree edge device. The multi-image registration fusion method comprises the following steps: (1) identifying image sharpness consistency; (2) identifying the image blurring balance; (3) extracting feature points of an effective region of the image; (4) constructing a geometric algebraic descriptor to carry out matching registration, and carrying out affine transformation and least square method fusion; according to the invention, the images at different visual angles can be accurately aligned, seamless fusion is carried out, and high quality and high readability of the final image are ensured.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a multi-image registration and fusion method for low-computation edge devices. Background Art

[0002] In traditional image processing technology, image information of the target area is usually obtained by shooting and processing at a single angle. However, for uneven surfaces, images from a single angle often cannot provide enough detailed information, especially in the presence of reflections or uneven light intensity. Therefore, multi-angle image acquisition and fusion has become an effective technical means to improve the quality and usability of the final image by shooting multiple times and combining image information from different angles.

[0003] In the prior art, some methods have attempted to improve image quality through multi-image registration and fusion. Existing image registration technologies are mainly based on methods such as feature point matching, image grayscale difference, and optical flow analysis. However, these methods generally rely on strong computing resources and are difficult to apply to low-computation edge devices, especially in handheld devices that require real-time processing and fast response. Existing multi-image fusion methods also rely on complex algorithms and high computing power, and are difficult to run efficiently on low-power, low-computation devices. In addition, existing image quality assessment methods mainly focus on the detection and optimization of indicators such as sharpness, contrast, and noise. However, for image registration and fusion technology for low-computation edge devices, how to effectively identify the effective area in the image and perform fast geometric registration and fusion while ensuring real-time performance is still a technical problem that needs to be solved urgently. Summary of the invention

[0004] Purpose of the invention: The purpose of the present invention is to provide a multi-image registration and fusion method for low-computation edge devices, which can quickly identify valid areas in the image through indicators such as sharpness consistency and blur balance, and perform registration and fusion based on the geometric features of these areas to solve the problems existing in the background technology.

[0005] Technical solution: The multi-image registration and fusion method for low-computation edge devices described in the present invention comprises the following steps:

[0006] (1) Identify image sharpness consistency;

[0007] (2) Identify the image blur balance;

[0008] (3) Extract feature points of the effective area of ​​the image;

[0009] (4) Construct a geometric algebraic descriptor for matching and registration and perform affine transformation and least squares fusion.

[0010] Furthermore, step (1) is specifically as follows: by calculating the local sharpness of each pixel in the image and combining the change of the image gradient, the stability of the sharpness in the region is evaluated; the local sharpness is measured by calculating the gradient of the image; the image is divided into several small regions, the sharpness consistency of each region is calculated, and the high-sharpness and consistent regions are identified as valid regions.

[0011] Furthermore, step (2) is specifically as follows: the degree of blur is evaluated using local high-frequency component analysis, the blur of each region is calculated, and the uniformity of the blur within the region is analyzed; wherein, regions with low and balanced blur are considered to be valid regions.

[0012] Furthermore, step (3) is specifically as follows: extracting feature points using the Harris corner detection algorithm; determining the response value of the corner point by calculating the gradient and autocorrelation matrix within each valid area, and selecting the point with a higher response value as the feature point.

[0013] Furthermore, in step (4), a descriptor is constructed and the similarity between feature points is calculated through an operation framework based on geometric algebra; then, the feature points in different images are matched through the minimum Euclidean distance or cosine similarity metric.

[0014] Furthermore, in step (4), the images are geometrically aligned using affine transformation; the affine transformation matrix, including translation, rotation and scaling, is calculated using the least squares method through matching feature point pairs; the transformation matrix is ​​accurately calculated by minimizing the error between the transformed image and the target image; finally, all images are registered using affine transformation, and weighted averaging is used to synthesize multiple aligned images into a high-quality image to complete image fusion.

[0015] The multi-image registration and fusion system for low-computation edge devices described in the present invention includes:

[0016] Sharpness module: used to identify the consistency of image sharpness;

[0017] Equalization module: used to identify the fuzzy balance of the image;

[0018] Extraction module: used to extract feature points in the effective area of ​​the image;

[0019] Fusion module: used to construct geometric algebraic descriptors for matching and registration and perform affine transformation and least squares fusion.

[0020] An electronic device described in the present invention includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the computer program is loaded into the processor, the electronic device implements any one of the multi-image registration and fusion methods for low-computational edge devices.

[0021] A storage medium described in the present invention stores a computer program, characterized in that when the computer program is executed by a processor, any one of the multi-image registration and fusion methods for low-computation edge devices is implemented.

[0022] Beneficial effects: Compared with the prior art, the present invention has the following significant advantages: The present invention significantly improves the image quality through precise multi-image registration and fusion methods, especially under different shooting angles, lighting changes and image deformation conditions, and can effectively extract clear and consistent image areas. Through multi-dimensional recognition of sharpness consistency, blur balance, brightness balance, and structural consistency, combined with efficient feature point matching and geometric algebra descriptor construction, this method can accurately align images from different perspectives and seamlessly fuse them, ensuring high quality and high readability of the final image. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 is a flow chart of the present invention;

[0024] Figure 2 is the scale neighborhood graph of the present invention;

[0025] Figure 3 is a feature descriptor matching graph of the present invention;

[0026] Figure 4 This is the final effect diagram of the present invention. DETAILED DESCRIPTION

[0027] The technical solution of the present invention is further described below in conjunction with the accompanying drawings.

[0028] like Figure 1 As shown, an embodiment of the present invention provides a multi-image registration and fusion method for a low-computation edge device, comprising the following steps:

[0029] (1) Identify the sharpness consistency of the image; evaluate the stability of the sharpness in the region by calculating the local sharpness of each pixel in the image and combining the change of the image gradient; measure the local sharpness by calculating the gradient of the image; divide the image into several small regions, calculate the sharpness consistency of each region, and identify the regions with high sharpness and consistency as valid regions. The specific process is as follows:

[0030] The sharpness consistency area in the image is identified by the local mean gradient. Unlike the traditional convolution operation, a simple local gradient change is used to calculate the sharpness of the image, and a measure of the local gradient variance is introduced to evaluate the sharpness consistency within the region. This method avoids complex convolution calculations and is suitable for implementation on low-computation edge devices.

[0031] Sharpness consistency means that the change in sharpness remains consistent within a local area of ​​the image, usually manifested as clear texture or details within the image area. Areas with lower sharpness values ​​generally indicate that the area is blurry, while areas with higher and consistent sharpness values ​​indicate clarity and stability.

[0032] (a) Local gradient calculation

[0033] In order to avoid complex convolution calculations, the central difference method is used to calculate the gradient of each pixel. Specifically, the gradient is estimated by calculating the difference between each pixel and its surrounding pixels. For a pixel I(x,y) in the image, the horizontal and vertical gradients can be approximately calculated as follows:

[0034] G x (x,y)=I(x+1,y)-I(x-1,y) (1)

[0035] G y (x,y)=I(x,y+1)-I(x,y-1) (2)

[0036] Then, the sharpness of the pixel can be measured by the gradient magnitude:

[0037] G(x,y)=G x (x,y) 2 +G y (x,y) 2 (3)

[0038] (b) Calculate the mean sharpness of the local area

[0039] Select a window size w×w (for example, 3×3 or 5×5) in the image, and for each pixel (x, y), calculate the mean sharpness in the window. The mean sharpness can be obtained by calculating the average value of the gradient amplitude of all pixels in the window:

[0040]

[0041] (c) Calculate the sharpness variance of the local area:

[0042] To evaluate the sharpness consistency, we calculate the sharpness variance within each local window as a measure of sharpness consistency.

[0043] A smaller sharpness variance indicates that the sharpness of the area remains consistent, whereas a larger sharpness variance indicates that the sharpness of the area varies greatly.

[0044]

[0045] (d) Sharpness consistency area determination:

[0046] According to the calculated sharpness mean and sharpness variance, we can set a threshold T to judge the sharpness consistency of the region. The specific judgment criteria are: if the sharpness mean μ of a window is G (x,y) is greater than a certain threshold T μ , and the sharpness variance Less than a certain threshold T σ , then the area is considered to have good sharpness consistency and can be used as a valid area.

[0047]

[0048] This method can effectively identify areas with consistent sharpness in the image, provide a basis for the final effective area determination, and has high practicality on low-computation devices.

[0049] (2) Identify the blur balance of the image; use local high-frequency component analysis to evaluate the blur degree, calculate the blur degree of each area, and analyze the uniformity of the blur degree within the area; among them, the area with low blur and balanced blur is considered to be a valid area. The specific process is as follows: estimate the blur degree of the image through local high-frequency component analysis, and calculate the consistency of the blur degree, that is, the stability of the blur degree in the same area. This method uses a local mean frequency analysis method with low computational complexity, avoids the complexity of frequency domain transformation, and is suitable for low-computation devices.

[0050] (S1) Calculation of local high-frequency components

[0051] In order to evaluate the blurriness, the local high-frequency component H(x,y) of each pixel is calculated. This value represents the degree of high-frequency change of the point in the image. The larger the high-frequency component, the clearer the image.

[0052] H(x,y)=G x (x,y)+G y (x,y) (7)

[0053] (S2) Local ambiguity calculation

[0054] Select a local region of size w×w (e.g. 3×3 or 5×5) in the image. For each local region R(x,y) in the image, the blurriness can be estimated by calculating the high-frequency mean in the region. The lower the high-frequency component in the region, the blurrier the region.

[0055]

[0056] Where N is the total number of pixels in the region R, and F(R) is the blur value of the region.

[0057] (S3) Fuzzy consistency

[0058] In a multi-region image, the blur consistency is evaluated by calculating the standard deviation of blur in different regions. If the blur of a local region is low (i.e., the region is clear) and consistent with the blur of its neighboring regions (i.e., the blur variation is small), then the region is considered clear and consistent. For multiple regions in the image, their blur variance is calculated.

[0059]

[0060] Where M is the total number of regions, F(R i ) is the fuzziness of the i-th region, is the average value of blur in all regions. If the blur variance is small, it means that the blur of the image is more consistent, otherwise, it means that the blur of the image is inconsistent.

[0061] (3) Extract feature points in the effective area of ​​the image; use the Harris corner detection algorithm to extract feature points; determine the response value of the corner point by calculating the gradient and autocorrelation matrix in each effective area, and select the point with a higher response value as the feature point. The specific process is as follows:

[0062] Compute the autocorrelation matrix:

[0063]

[0064] Then calculate the response function based on M:

[0065] R = det(M) - k(trace(M)) 2 (11)

[0066] Where det(M) is the determinant of the matrix, trace(M) is the trace of the matrix, and k is a constant, usually between 0.04 and 0.06. Threshold processing is performed on the response function, and points with larger response values ​​are selected as corner points.

[0067] (4) Construct geometric algebraic descriptors for matching and registration and perform affine transformation and least squares fusion. Here, descriptors are constructed and the similarity between feature points is calculated through an operation framework based on geometric algebra; then, feature points in different images are matched using the minimum Euclidean distance or cosine similarity metric. The specific process is as follows:

[0068] The calculated feature point set is recorded as I sum is the total number of images, H m Any feature point in sumP m is the total number of feature points of image m. For feature point p m,iThe scale neighborhood A = 15 × 15 × 15, all feature points in the neighborhood are expressed in geometric algebraic space as

[0069]

[0070] At this time, the feature point is represented as a vector v with direction and magnitude m,i The present invention uses the characteristics of geometric algebra inner product that can measure vector angle and projection intensity to construct each feature point p m,i The feature descriptor of . Assume that from p m,i Starting from this, we can derive a unit vector uv i =x i e 1 +y i e 2 , calculate the unit vector uv i With p m,i The sum of the inner products of all feature points in the scale neighborhood A To solve S i The maximum value is the objective function to construct a single Lagrangian function with equal constraints, and its constraints are uv i is the unit vector ||x i e 1 +y i e 2 ||=1, the objective function is expressed as:

[0071]

[0072] For x in the above formula i ,y i and λ i Taking partial derivatives in turn, we get:

[0073]

[0074] The simultaneous equations give:

[0075]

[0076] According to S 1 The second-order Hessian matrix of is negative definite, the extreme point calculated by equation (15) is the maximum value, and the unique direction vector uv is i , at this time uv i It is rotationally invariant and affine invariant.

[0077] Reconstruction p m,i of Scale neighborhood. Figure 2 As shown, uv i For the positive direction, construct each C i,kThe characteristic direction is calculated in the same way as the main direction. The 64 main directions constructed are p m,i The feature descriptor D m,i .

[0078] Use affine transformation to geometrically align images; use the least squares method to calculate the affine transformation matrix, including translation, rotation, and scaling, through matching feature point pairs; accurately calculate the transformation matrix by minimizing the error between the transformed image and the target image; finally, align all images through affine transformation, and use weighted averaging to synthesize multiple aligned images into a high-quality image to complete image fusion. The specific process is as follows:

[0079] (A) Feature point descriptor matching

[0080] Feature point matching is performed by calculating the similarity of the feature point descriptors between each image. Given a feature descriptor D m,i The random consistency sampling algorithm is used to optimize the descriptor, remove the outliers in the matching samples, reduce mismatches, and improve the accuracy of descriptor matching. m,i The magnitude of the value is affected by the scale field C i,k Due to the influence of the gradient size, the scale of the descriptor is quite different, and the traditional Euclidean distance cannot describe the similarity of the descriptor; while the Mahalanobis distance can describe the similarity of two unknown sample sets independently of the scale, and correct the problems such as the inconsistent scale of each dimension in the Euclidean distance. Therefore, the Mahalanobis distance is used to represent the two feature descriptors, which is:

[0081]

[0082] Among them, D k,i represents the i-th descriptor of the k-th image, D k-1,j represents the jth descriptor of the k-1th image, Σ -1 is the covariance matrix of the multidimensional random variable. Using the Mahalanobis distance as the matching measure of the descriptor, after the overall sorting by matching rate, the later the descriptor pairs are, the higher the error or mismatch rate will be. Therefore, usually only the descriptor pairs with high matching rate are used in the precise registration calculation. The matching effect is as follows Figure 3 shown.

[0083] (B) Affine transformation estimation

[0084] For each pair of matching feature points (p k,i ,p k-1,j ), and use it as the corresponding point to calculate the affine transformation. Affine transformation includes translation, rotation, scaling and shearing operations. Given a feature point p k,i =(x k,i ,y k,i ) and pk-1,j =(x k-1,j ,y k-1,j ), the affine transformation can be expressed as:

[0085]

[0086] Use the least squares method to estimate the parameters of the affine transformation matrix {a 11 ,a 12 ,a 21 ,a 22 ,t x ,t y}, which can be solved by linear equations.

[0087] (C) Least Squares Fusion

[0088] After completing the affine transformation and deformation estimation, the next step is image registration and fusion. The effective areas of multiple images are fused by the least squares fitting method. For the overlapping areas, the Gaussian weighted average method is used to avoid obvious transitions at the fusion edge while maintaining the consistency of the image content:

[0089]

[0090] where w k (x,y) and w k-1 (x,y) is the weighting factor for each image, depending on the overlapping area of ​​the images.

[0091] Through these steps, the present invention provides an efficient multi-image registration and fusion method. Through effective area recognition, feature point extraction and matching, and descriptor construction based on geometric algebra, combined with affine transformation and least squares fitting, accurate image alignment and fusion are achieved. This method is not only highly accurate, but also has excellent computational efficiency, is suitable for low-computation devices, and has broad application prospects.

[0092] like Figure 4 As shown, the final effect shows that the image is significantly improved after registration and fusion, especially in terms of detail clarity and texture fidelity, which has been greatly optimized.

Claims

1. A multi-image registration and fusion method for low-computation edge devices, characterized in that: The following steps are involved: (1) Identify image sharpness consistency; (2) Identify the image blur balance; (3) Extract feature points in the effective area of ​​the image; (4) Construct a geometric algebraic descriptor for matching and registration and perform affine transformation and least squares fusion.

2. The multi-image registration and fusion method for low-computation edge devices according to claim 1, characterized in that: Step (1) is as follows: by calculating the local sharpness of each pixel in the image and combining it with the change of the image gradient, the stability of the sharpness in the region is evaluated; the local sharpness is measured by calculating the gradient of the image; the image is divided into several small regions, the sharpness consistency of each region is calculated, and the high-sharp and consistent regions are identified as valid regions.

3. The multi-image registration and fusion method for low-computation edge devices according to claim 1, characterized in that: Step (2) is as follows: the blur degree is evaluated using local high-frequency component analysis, the blur degree of each region is calculated, and the uniformity of the blur degree within the region is analyzed; among them, the region with low and balanced blur degree is considered to be a valid region.

4. The multi-image registration and fusion method for low-computation edge devices according to claim 1, characterized in that: Step (3) is as follows: extract feature points using the Harris corner detection algorithm; determine the response value of the corner point by calculating the gradient and autocorrelation matrix in each valid area, and select the point with a higher response value as the feature point.

5. The multi-image registration and fusion method for low-computation edge devices according to claim 1, characterized in that: In step (4), a descriptor is constructed and the similarity between feature points is calculated through an operation framework based on geometric algebra; then, the feature points in different images are matched using the minimum Euclidean distance or cosine similarity metric.

6. The multi-image registration and fusion method for low-computation edge devices according to claim 1, characterized in that: In step (4), the images are geometrically aligned using affine transformation; the affine transformation matrix, including translation, rotation and scaling, is calculated using the least squares method through matching feature point pairs; the transformation matrix is ​​accurately calculated by minimizing the error between the transformed image and the target image; finally, all images are registered using affine transformation, and multiple aligned images are synthesized into a high-quality image using weighted averaging to complete image fusion.

7. A multi-image registration and fusion system for low-computation edge devices, characterized in that: include: Sharpness module: used to identify the consistency of image sharpness; Equalization module: used to identify the fuzzy balance of the image; Extraction module: used to extract feature points in the effective area of ​​the image; Fusion module: used to construct geometric algebraic descriptors for matching and registration and perform affine transformation and least squares fusion.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the computer program is loaded into the processor, it implements the multi-image registration and fusion method for low-computation edge devices according to any one of claims 1 to 6.

9. A storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, a multi-image registration and fusion method for a low-computation edge device according to any one of claims 1 to 6 is implemented.

Images