High-efficiency binocular camera splicing method

Through memory copy and nonlinear mapping, the binocular camera stitching process is optimized, and the problems of high CPU occupancy and field angle loss during high-resolution image processing are solved, and binocular camera stitching with high efficiency and diverse functions are achieved.

CN119090711APending Publication Date: 2024-12-06INGENIC SEMICON CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202310635244.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-05-31
Publication Date
2024-12-06

AI Technical Summary

Technical Problem

When processing high-resolution images, the binocular camera stitching method has a high CPU occupancy rate, cannot process data in real time, and has a large loss in field-of-view angle, which limits the product functions and usage range.

Method used

Memory copy technology and nonlinear mapping are used, combining perspective transformation and multi-band mixing, optimize the image stitching process, reduce CPU occupancy and ensure a large field of view angle. Specific steps include obtaining real-time frames, perspective transformation, memory copying, nonlinear mapping and multi-band mixing, through these steps, achieving efficient binocular camera splicing.

Benefits of technology

It significantly reduces the CPU usage, gives enough resources to run other algorithms, such as face recognition and humanoid detection, and ensures a large field of view angle, improving the efficiency and functional diversity of binocular camera splicing.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119090711A_ABST
    Figure CN119090711A_ABST
Patent Text Reader

Abstract

The invention provides a high-efficiency binocular camera splicing method. The method comprises the following steps: S1, obtaining real-time frames: a left camera real-time frame and a right camera real-time frame; s2, after a result of the step S1 is obtained, reading and storing a checkerboard four-vertex coordinate file; s3, judging whether vertex coordinates are obtained or not; if not, performing the step S4; if yes, the step S5 is carried out; s4, calibrating the camera, and obtaining internal parameters and distortion parameters of the camera; checkerboard detection: speculating coordinates of four vertexes of a checkerboard; returning to the step S2; s5, performing perspective transformation; s6, memory copy is carried out; in the step, the left half part of the left camera image and the right half part of the right camera image are copied to the left end and the right end of the spliced image respectively, and perspective transformation is avoided in the process; s7, performing nonlinear mapping; and S8, carrying out multi blow. According to the method, the memory copy technology is utilized, the binocular splicing operation efficiency is improved, the CPU occupancy rate during high-resolution image processing is reduced, and enough CPU resources are provided for operation of other algorithms. And the requirement of a relatively large field angle is ensured.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention belongs to the technical field of image processing, and in particular relates to a high-efficiency binocular camera stitching method. Background Art

[0002] In the prior art, binocular camera image stitching is to stitch images taken by two cameras at different positions at the same time. Images with partial scene overlap are stitched into an image with a wide viewing angle. Usually, image stitching generally includes three parts: warp (mapping), compensation (light compensation) and blend (fusion). The warp part mainly depends on the accuracy of camera parameter estimation, the light compensation is mainly used to solve the illumination difference of different parts of the output image caused by different exposure of different images, and the blend is used to fuse the overlapping parts between different images, and generally a linear weighted method is used to obtain the final output image. The basic idea of ​​multi-band blending is that the image can be decomposed into a superposition of images of different frequencies (similar to Fourier transform). At different frequencies, different weights should be used for fusion. In the low-frequency part, a weighted signal with a wider wavelength should be used (for example, the sigma in the Gaussian kernel function is relatively large), and in the high-frequency part, a narrower weighted signal should be used (for example, the sigma of the Gaussian kernel function is relatively small). The algorithm is as follows:

[0003] 1. Calculate the Gaussian pyramid of the input image. If the input image is A, B, then calculate G A0 ,G A1 ,G A2 ,G A3 ,… and G B0 ,G B1 ,G B2 ,G B3 ,……

[0004] The basic operations of the image pyramid are:

[0005] An image is continuously blurred and then downsampled to obtain images of different resolutions. At the same time, the width and height of each new image is 1 / 2 of the original image. The most common method is to sample after Gaussian blurring. The resulting series of images is called a Gaussian pyramid.

[0006] 2. Calculate the Laplacian pyramid of the input image. Note it as L A0 ,L A1 ,L A2 ,L A3 ,… and L B0 ,L B1 ,L B2 ,L B3 ,……

[0007] The Laplace pyramid is calculated as follows:

[0008] L(i)=G(i)–expand(G(i+1))

[0009] The i-th Laplacian pyramid is obtained by subtracting the i+1-th Gaussian pyramid expand from the i-th Gaussian pyramid.

[0010] 3. Fuse the Laplacian pyramids at the same level. For example, use simple linear fusion on both sides of the seam. Let the output image be C, then we get L C0 ,L C1 ,……

[0011] 4. Expand the high-level Laplacian pyramid in sequence until it has the same resolution as LC0. Recorded as L C00 ,L C11 ,L C22 …

[0012] 5. Superimpose the images obtained in 4 in sequence to obtain the final output image.

[0013] However, the defects in the existing technology are: when processing high-resolution images, the CPU usage rate of the existing image stitching method has reached about 90% at low frame rates, and it is unable to process real-time data and run other algorithms, such as face recognition and human shape detection. As a result, the product has a single function and a limited scope of use. In addition, the field of view angle is greatly lost before and after stitching, and a large field of view angle cannot be guaranteed.

[0014] In addition, commonly used technical terms in the prior art include:

[0015] 1. Binocular camera stitching: stitch and merge the photos taken by the left and right cameras placed at a certain angle into one photo.

[0016] 2. Camera calibration: In the image measurement process and machine vision applications, in order to determine the relationship between the three-dimensional geometric position of a point on the surface of a spatial object and its corresponding point in the image, a geometric model of camera imaging must be established. These geometric model parameters are camera parameters. Under most conditions, these parameters must be obtained through experiments and calculations. The process of solving the parameters is called camera calibration (or camera calibration).

[0017] 3. memory copy: memory copy.

[0018] 4. Multi blend: The idea of ​​multi-band fusion is to construct Laplacian pyramids for the fused images (Laplacian operator can extract high-frequency information of images. In the Laplacian pyramid, the higher the image layer, the higher the frequency). Then, the images in the same layer are fused according to certain rules, usually Alpha blending. Different rules are used to fuse images of different layers (images of different frequency bands), and the fused image pyramid is reconstructed to obtain the final result image. Summary of the invention

[0019] In order to solve the above problems, the purpose of this application is: This method uses memory copy technology to improve the efficiency of binocular stitching operation, reduce its CPU occupancy when processing high-resolution images, and have enough CPU resources to run other algorithms. In addition, there is no pixel loss in memory copy, which ensures a larger field of view angle requirement. Specifically, the present invention provides a high-efficiency binocular camera stitching method, which includes the following steps:

[0020] S1, obtain real-time frames: left camera real-time frame, right camera real-time frame;

[0021] S2, after obtaining the result of step S1, read the config.json file that stores the coordinates of the four vertices of the chessboard;

[0022] S3, determine whether the vertex coordinates are obtained? If not, proceed to step S4; if yes, proceed to step S5;

[0023] S4, camera calibration, obtaining camera internal parameters and distortion parameters; chessboard detection, inferring the coordinates of the four vertices of the chessboard; returning to step S2;

[0024] S5, perspective transformation:

[0025] The coordinates of the four vertices of the chessboard detected in the left and right camera images are weighted averaged and scaled to generate new coordinates of the four vertices of the chessboard; further comprising:

[0026] S5.1, a total of three perspective transformations are performed during the stitching process. The first and second perspective transformations are respectively performed to obtain the perspective transformation matrices from the checkerboards in the left and right camera images to the new checkerboards, so that the right half of the left camera image and the left half of the right camera image are mapped to a temporary puzzle image. The perspective transformation matrix of the left / right image mapped to the stitched image is obtained according to the four vertex coordinates of the left / right checkerboards and the four vertex coordinates of the new checkerboards;

[0027] S5.2, the third perspective transformation is to transform the temporary puzzle in the previous step S5.1 to the mosaic image. This step is to ensure that the transition between the memory copy part image in step S6 and the perspective transformation image is more natural. The specific operation method is the same as that in the previous step S5.1.

[0028] S6, memory copy;

[0029] This step is to copy the left half of the left camera image and the right half of the right camera image to the left and right ends of the mosaic respectively. This process avoids the use of perspective transformation; further includes:

[0030] S6.1, in the memory storing the left image, sequentially copy the memory data of the left half row by row, and then sequentially paste the copied data by row from the beginning of the mosaic image row until all the copied data are pasted into the mosaic image;

[0031] S6.2, in the memory storing the right image, copy the memory data of the right half row row by row and arrange the data in reverse order, and then paste the copied data row by row starting from the end of the mosaic image row in reverse order until all the copied data are pasted into the mosaic image;

[0032] S7, nonlinear mapping;

[0033] Since the image after the third perspective is not in proportion to the memory copy images on the left and right ends, and there are missing pixels on the upper and lower edges of the perspective transformation part, nonlinear mapping is used to compress the image on the left and right and stretch the image on the top and bottom to solve this problem;

[0034] S8, multi-band blending, the specific process is as follows:

[0035] S8.1, along the height direction, take 24 pixels on the left and right sides of the stitching position in the left and right mapping images respectively as the left and right images to be fused in the multi blend;

[0036] S8.2, construct the Laplacian pyramid of the left and right images to be fused, and then construct the masked Gaussian pyramid as the weight coefficient of weighted fusion;

[0037] S8.3, multiply the Laplacian pyramids of the left and right images by the weight coefficients respectively, and sum them up to obtain the final fused image pyramid;

[0038] S8.4, reconstructing a final fused image using the fused image pyramid; and placing the fused image back to the splicing position in the splicing image;

[0039] S8.5, final output effect.

[0040] In step S4:

[0041] The camera calibration: calibrate the binocular cameras respectively to obtain the camera lens distortion parameters, that is, use the coefficients a and b in the following formula;

[0042] y=a*arctan(b*x)

[0043] Where y is the distance from a certain pixel in the correction image to the center of the correction image, and x is the distance from the corresponding pixel in the distortion image to the center of the distortion image. For the same model of lens, it only needs to be calibrated once, which is the data preparation before image stitching;

[0044] The chessboard detection:

[0045] When the stereo camera is initialized, it is necessary to detect the coordinates of the corner points of the chessboard in the stitching image and save them in the configuration file. When the stereo camera is restarted later, it does not need to detect the chessboard again, but reads it directly from the configuration file.

[0046] In step S5, the calculation formula used in the detection process is as follows:

[0047] new_chessboard_x=(left_x+right_x+img_width) / 2

[0048] new_chessboard_y=(left_y+right_y) / 2

[0049] Among them, new_chessboard_x and new_chessboard_y are the x and y coordinates of the vertices of the new chessboard, left_x and left_y are the x and y coordinates of the vertices of the chessboard captured by the left camera, right_x and right_y are the x and y coordinates of the vertices of the chessboard captured by the right camera, and img_width is the width of the pictures taken by the left and right cameras.

[0050] In step S5.1, the specific perspective transformation formula used in the operation method is:

[0051]

[0052]

[0053] Where X i , Y i is the coordinate of the i-th vertex among the four vertices of the new chessboard; a ij are the jth element values ​​of the ith row in the perspective transformation matrix of size 3×3, where a 33 is a constant value 1, and the rest are values ​​to be evaluated; x i ,yi is the coordinate of the i-th vertex among the 4 vertices of the left / right chessboard. Solve a based on the coordinates of the 4 vertices of the left / right chessboard and the 4 vertices of the new chessboard. ij Value, get the perspective transformation matrix that maps the left / right image to the stitching image.

[0054] In step S7, the formula used by the nonlinear mapping to compress the image left and right and stretch the image up and down is as follows:

[0055] x i ′=img_width+a1*tan(a2*(x i -width)*π)

[0056]

[0057] Among them, img_width and img_height are the width and height of the left and right camera images, x i 、x i ′ are the X-axis coordinates of the pixels before and after mapping, and y i ,y i ′ is the Y-axis coordinate of the pixel before and after mapping; width is the width of the image after mapping; a i is the corresponding nonlinear mapping coefficient.

[0058] In step S8.3, the formula used is as follows:

[0059] p i,j,k =pl i,j,k *w i,j,k +pr i,j,k *(1-w i,j,k )

[0060] where p i,j,k ,pl i,j,k ,pr i,j,k are the pixel values ​​of the fused image pyramid, the Laplacian pyramid on the left, and the Laplacian pyramid on the right, i.e., the jth row and the kth column. i,j,k is the weight value of the jth row and kth column of the i-th layer of the masked Gaussian pyramid.

[0061] Therefore, the advantages of the present application are: the method is simple and can well solve the problems in the prior art. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] The drawings described herein are used to provide a further understanding of the present invention, constitute a part of this application, and do not constitute a limitation of the present invention.

[0063] Figure 1 is a schematic diagram of the process of this method.

[0064] Figure 2 It is a schematic diagram of image capture by the left camera in this method.

[0065] Figure 3 It is a schematic diagram of image capture by the right camera in this method.

[0066] Figure 4 It is a schematic diagram of three perspective transformations performed in this method.

[0067] Figure 5 This is a schematic diagram of the image after optimization by this method. DETAILED DESCRIPTION

[0068] In order to more clearly understand the technical content and advantages of the present invention, the present invention is now further described in detail in conjunction with the accompanying drawings.

[0069] like Figure 1 As shown, the present application provides a high-efficiency binocular camera stitching method, such as Figure 1 As shown,

[0070] S0, start;

[0071] S1, left camera real-time frame, right camera real-time frame;

[0072] S2, after obtaining the result of S1, read and save the coordinate file of the four vertices of the chessboard;

[0073] S3, determine whether the vertex coordinates are obtained? If not, proceed to step S4; if yes, proceed to step S5;

[0074] S4, camera calibration, obtaining camera internal parameters and distortion parameters; chessboard detection, inferring the coordinates of the four vertices of the chessboard; returning to step S2;

[0075] The camera calibration:

[0076] Calibrate the binocular cameras separately to obtain the camera lens distortion parameters. The same model of lens only needs to be calibrated once, which is the data preparation before image stitching.

[0077] The checkerboard detection

[0078] When the stereo camera is initialized, it is necessary to detect the coordinates of the corner points of the chessboard in the stitching image and save them in the configuration file. When the stereo camera is restarted later, it does not need to detect the chessboard again, but directly reads from the configuration file. The actual shooting picture is as follows Figure 2 and Figure 3 shown.

[0079] S5, perspective transformation:

[0080] The four vertex coordinates of the chessboard detected in the left and right camera images are weighted averaged and scaled to generate new coordinates of the four vertices of the chessboard. The formula used is as follows:

[0081] new_chessboard_x=(left_x+right_x+img_width) / 2

[0082] new_chessboard_y=(left_y+right_y) / 2

[0083] Among them, new_chessboard_x and new_chessboard_y are the x and y coordinates of the vertices of the new chessboard, left_x and left_y are the x and y coordinates of the vertices of the chessboard captured by the left camera, right_x and right_y are the x and y coordinates of the vertices of the chessboard captured by the right camera, and img_width is the width of the pictures taken by the left and right cameras.

[0084] A total of three perspective transformations are performed during the stitching process, such as Figure 4 As shown. The first and second times are to calculate the perspective transformation matrix from the chessboard to the new chessboard in the left and right camera images respectively, so that the right half of the left camera image and the left half of the right camera image are mapped to a temporary image. The specific formula used for perspective transformation is:

[0085]

[0086]

[0087] Where X i , Y i is the coordinate of the i-th vertex among the four vertices of the new chessboard; a ij are the jth element values ​​of the ith row in the perspective transformation matrix of size 3×3, where a33 is a constant value 1, and the rest are values ​​to be evaluated. i ,y i is the coordinate of the i-th vertex among the 4 vertices of the left / right chessboard. Solve a based on the coordinates of the 4 vertices of the left / right chessboard and the 4 vertices of the new chessboard. ij Value, get the perspective transformation matrix that maps the left / right image to the stitching image.

[0088] The third perspective transformation is to transform the temporary puzzle from the previous step to the spliced ​​image. This step is to ensure that the transition between the memory copy image and the perspective transformed image is more natural. The specific calculation process is the same as the previous step.

[0089] S6, memory copy;

[0090] This step is to copy the left half of the left camera image and the right half of the right camera image to the left and right ends of the mosaic image respectively. Figure 4 This process avoids the use of perspective transformation, greatly reduces the amount of calculation, and improves efficiency by about 45%.

[0091] S6.1, in the memory storing the left image, copy the memory data of the left half row in sequence by row, and then paste the copied data in sequence by row from the beginning of the mosaic image row until all the copied data are pasted into the mosaic image.

[0092] S6.2, in the memory storing the right image, copy the memory data of the right half row row by row and arrange the data in reverse order, and then paste the copied data row by row in reverse order starting from the end of the mosaic image row until all the copied data are pasted into the mosaic image.

[0093] S7, nonlinear mapping;

[0094] Because the image after the third perspective is not in proportion to the memory copy images on the left and right ends, and there are missing pixels on the upper and lower edges of the perspective transformation part. To solve this problem, nonlinear mapping is used to compress the image left and right and stretch the image up and down, such as Figure 4 The formula used is as follows:

[0095] x i ′=img_width+a1*tan(a2*(x i -width)*π)

[0096]

[0097] Among them, img_width and img_height are the width and height of the left and right camera images, x i 、x i ′ are the X-axis coordinates of the pixels before and after mapping, and y i ,y i ′ are the Y-axis coordinates of the pixels before and after mapping. width is the width of the image after mapping. i is the corresponding nonlinear mapping coefficient.

[0098] S8, multi blend; pixel weighted fusion

[0099] The specific process of multi blend is as follows:

[0100] S8.1, along the height direction, take 24 pixels on the left and right sides of the stitching position in the left and right mapping images respectively as the left and right images to be fused in the multi blend.

[0101] S8.2, construct the Laplacian pyramid of the left and right images to be fused, and then construct the masked Gaussian pyramid as the weight coefficient for weighted fusion.

[0102] S8.3, multiply the Laplacian pyramids of the left and right images by the weight coefficients respectively, and sum them to obtain the final fused image pyramid. The formula used is as follows:

[0103] p i,j,k =pl i,j,k *w i,j,k +pr i,j,k *(1-w i,j,k )

[0104] where p i,j,k ,pl i,j,k ,pr i,j,k are the pixel values ​​of the fused image pyramid, the Laplacian pyramid on the left, and the Laplacian pyramid on the right, i.e., the jth row and the kth column. i,j,k is the weight value of the jth row and kth column of the i-th layer of the masked Gaussian pyramid.

[0105] S8.4, using the fused image pyramid, reconstruct the final fused image, and put the fused image back to the splicing position in the splicing image.

[0106] S8.5, the final output effect is as follows Figure 5 shown.

[0107] S9, end.

[0108] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the embodiments of the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A high-efficiency binocular camera stitching method, It is characterized in that The method comprises the following steps: S1, obtain real-time frames: left camera real-time frame, right camera real-time frame; S2, after obtaining the result of step S1, read the config.json file that stores the coordinates of the four vertices of the chessboard; S3, determine whether the vertex coordinates are obtained? If not, proceed to step S4; if yes, proceed to step S5; S4, camera calibration, obtaining camera internal parameters and distortion parameters; chessboard detection, inferring the coordinates of the four vertices of the chessboard; returning to step S2; S5, perspective transformation: The coordinates of the four vertices of the chessboard detected in the left and right camera images are weighted averaged and scaled to generate new coordinates of the four vertices of the chessboard; further comprising: S5.1, a total of three perspective transformations are performed during the stitching process. The first and second perspective transformations are respectively performed to obtain the perspective transformation matrices from the checkerboards in the left and right camera images to the new checkerboards, so that the right half of the left camera image and the left half of the right camera image are mapped to a temporary puzzle image. The perspective transformation matrix of the left / right image mapped to the stitched image is obtained according to the four vertex coordinates of the left / right checkerboards and the four vertex coordinates of the new checkerboards; S5.2, the third perspective transformation is to transform the temporary puzzle in the previous step S5.1 to the mosaic image. This step is to ensure that the transition between the memory copy part image in step S6 and the perspective transformation image is more natural. The specific operation method is the same as that in the previous step S5.

1. S6, memory copy; This step is to copy the left half of the left camera image and the right half of the right camera image to the left and right ends of the mosaic respectively. This process avoids the use of perspective transformation; further includes: S6.1, in the memory storing the left image, sequentially copy the memory data of the left half row by row, and then sequentially paste the copied data by row from the beginning of the mosaic image row until all the copied data are pasted into the mosaic image; S6.2, in the memory storing the right image, copy the memory data of the right half row row by row and arrange the data in reverse order, and then paste the copied data row by row starting from the end of the mosaic image row in reverse order until all the copied data are pasted into the mosaic image; S7, nonlinear mapping; Since the image after the third perspective is not in proportion to the memory copy images on the left and right ends, and there are missing pixels on the upper and lower edges of the perspective transformation part, nonlinear mapping is used to compress the image on the left and right and stretch the image on the top and bottom to solve this problem; S8, multi-band blending, the specific process is as follows: S8.1, along the height direction, take 24 pixels on the left and right sides of the splicing position in the left and right mapping images respectively as the left and right images to be fused in multiblend; S8.2, construct the Laplacian pyramid of the left and right images to be fused, and then construct the masked Gaussian pyramid as the weight coefficient of weighted fusion; S8.3, multiply the Laplacian pyramids of the left and right images by the weight coefficients respectively, and sum them up to obtain the final fused image pyramid; S8.4, reconstructing a final fused image using the fused image pyramid; and placing the fused image back to the splicing position in the splicing image; S8.5, final output effect.

2. A high-efficiency binocular camera stitching method according to claim 1, It is characterized in that In step S4: The camera calibration: calibrate the binocular cameras respectively to obtain the camera lens distortion parameters, that is, use the coefficients a and b in the following formula; y=a*arctan(b*x) Where y is the distance from a certain pixel in the correction image to the center of the correction image, and x is the distance from the corresponding pixel in the distortion image to the center of the distortion image. For the same model of lens, it only needs to be calibrated once, which is the data preparation before image stitching; The chessboard detection: when the binocular camera is initialized, the coordinates of the corner points of the chessboard in the spliced ​​image need to be detected and saved in the configuration file; when the binocular camera is restarted later, there is no need to detect the chessboard again, but to read it directly from the configuration file.

3. A high-efficiency binocular camera stitching method according to claim 1, It is characterized in that In step S5, the calculation formula used in the detection process is as follows: new_chessboard_x=(left_x+right_x+img_width) / 2 new_chessboard_y=(left_y+right_y) / 2 Among them, new_chessboard_x and new_chessboard_y are the x and y coordinates of the vertices of the new chessboard, left_x and left_y are the x and y coordinates of the vertices of the chessboard captured by the left camera, right_x and right_y are the x and y coordinates of the vertices of the chessboard captured by the right camera, and img_width is the width of the pictures taken by the left and right cameras.

4. A high-efficiency binocular camera stitching method according to claim 3, It is characterized in that In step S5.1, the specific perspective transformation formula used in the operation method is: Where X i , Y i is the coordinate of the i-th vertex among the four vertices of the new chessboard; a ij are the jth element values ​​of the ith row in the perspective transformation matrix of size 3×3, where a 33 is a constant value 1, and the rest are values ​​to be evaluated; x i ,y i is the coordinate of the i-th vertex among the 4 vertices of the left / right chessboard. Solve a based on the coordinates of the 4 vertices of the left / right chessboard and the 4 vertices of the new chessboard. ij Value, get the perspective transformation matrix that maps the left / right image to the stitching image.

5. The high-efficiency binocular camera stitching method according to claim 1, It is characterized in that In step S7, the formula used by the nonlinear mapping to compress the image left and right and stretch the image up and down is as follows: x i ′=img_width+a 1 *tan(a 2 *(x i -width)*π) Among them, img_width and img_height are the width and height of the left and right camera images, x i 、x i ′ are the X-axis coordinates of the pixels before and after mapping, and y i ,y i ′ is the Y-axis coordinate of the pixel before and after mapping; width is the width of the image after mapping; a i is the corresponding nonlinear mapping coefficient.

6. A high-efficiency binocular camera stitching method according to claim 5, It is characterized in that In step S8.3, the formula used is as follows: p i,j,k =pl i,j,k *w i,j,k +pr i,j,k *(1-w i,j,k ) where p i,j,k ,pl i,j,k ,pr i,j,k are the pixel values ​​of the fused image pyramid, the Laplacian pyramid on the left, and the Laplacian pyramid on the right, i.e., the jth row and the kth column. i,j,k is the weight value of the i-th layer, j-th row, and k-th column of the masked Gaussian pyramid.