Microscopic image splicing method based on improved linear fusion

Abstract

The invention discloses a microscopic image splicing method based on improved linear fusion, which comprises the steps of scanning and shooting, splicing calculation and image fusion, and has the advantages that: in an image fusion link of microscopic image splicing, after coordinate information of all images is obtained, according to a pixel-level image position relationship, the image fusion of the microscopic images is realized; image splicing of a non-overlapping area, image fusion of an overlapping area of two images and image fusion of an overlapping area of four images are sequentially carried out on each row of images, so that the overlapping areas of the images are in natural transition, image brightness transition is natural, the problem of splicing marks caused by uneven illumination is solved, and the image quality is improved. The method is particularly suitable for solving the problem of splicing marks existing when cross overlapping areas of four adjacent images are processed through conventional linear fusion. In the image fusion process, the multi-thread technology is adopted, the image fusion efficiency of the overlapping area is improved, the operation time is saved, and the average acceleration rate is about 2.74 times.

Description

Technical Field

[0001] The invention relates to a microscope image stitching method, in particular to a microscope image stitching method based on improved linear fusion. Background Art

[0002] Microscopic image stitching is a common microscopic image technology that can expand the microscopic field of view. This technology mainly includes three steps: scanning and shooting, stitching calculation, and image fusion. If the illumination uniformity of the microscope is not good, it is easy to cause uneven image brightness, which in turn leads to abnormal stitching in the image fusion process.

[0003] Patent application number CN11023273B introduces a linear fusion method, which sets weight parameters based on distance relationships and processes pixels in the overlapping area of ​​two images. This method is a common microscopic image fusion method and is suitable for microscopic image fusion scenes with uniform or slightly uneven illumination brightness. However, when the illumination brightness is obviously uneven and the microscopic optical path system cannot be changed, the use of the linear fusion method in the cross-overlapping area of ​​four adjacent images is prone to produce splicing marks. Summary of the invention

[0004] The technical problem to be solved by the present invention is to provide a microscopic image stitching method based on improved linear fusion, which can eliminate image stitching marks caused by uneven illumination brightness.

[0005] The technical solution adopted by the present invention to solve the above technical problems is: a method for stitching microscopic images based on improved linear fusion, including scanning and shooting, stitching calculation and image fusion, wherein the image fusion comprises the following steps:

[0006] Step (1): Set the total number of scanned images to ROW and COL, set the position of each image to the number "row number_column number", record the row number as row, row∈[1,ROW], and record the column number as col, col∈[1,COL];

[0007] Step (2): According to the result of the splicing calculation, the coordinate matrix of each image is obtained, denoted as P;

[0008] Step (3): Calculate the relative position offset matrix between each image and the adjacent upper image and the adjacent left image, denoted as M;

[0009] Step (4): Use multi-threading technology to achieve image fusion and complete the stitching of microscopic images. The specific method is as follows:

[0010] Step (4-1): Create a blank image and use it as the current image, initialize the row number, and set row = 1;

[0011] Step (4-2): Determine whether the row number row meets the condition row≤ROW. If so, execute step (4-3); otherwise, execute step (4-8);

[0012] Step (4-3): Use the current large image as the original large image and add 1 to the row number row;

[0013] Step (4-4): according to the coordinate matrix P and the relative position offset matrix M, stitch the images of the non-overlapping areas of the COL images in the row th row to obtain the first partial image;

[0014] Step (4-5): According to the coordinate matrix P and the relative position offset matrix M, image fusion is performed on the overlapping area of ​​the two images in the row row COL images to obtain the second part of the image;

[0015] Step (4-6): According to the coordinate matrix P and the relative position offset matrix M, image fusion is performed on the overlapping area of ​​the four images in the row-th row COL images to obtain the third partial image;

[0016] Step (4-7): assigning the first partial image, the second partial image and the third partial image to the original large image to obtain the current large image, and returning to step (4-2);

[0017] Step (4-8): Use the current large image as the stitched microscopic image.

[0018] Compared with the prior art, the advantage of the present invention is that in the image fusion link of microscopic image stitching, after obtaining the coordinate information of all images, according to the image position relationship at the pixel level, the image stitching of non-overlapping areas, the image fusion of two-image overlapping areas, and the image fusion of four-image overlapping areas are carried out for each row of images in sequence, so that the overlapping areas of the images transition naturally, the image brightness transitions naturally, and the stitching mark problem caused by uneven lighting is improved. It is particularly suitable for solving the stitching mark problem existing in conventional linear fusion when processing the cross-overlapping areas of four adjacent images. In addition, in the image fusion process, the present invention adopts multi-threading technology to improve the image fusion efficiency of the overlapping areas and save running time. The present invention is applicable to different microscopic lighting conditions, such as wide-field microscopes and fluorescence microscopes.

[0019] Experimental results show that compared with the traditional linear fusion method, the present invention not only eliminates the image stitching problem caused by uneven lighting brightness, improves the shortcomings of the linear fusion method, but also significantly improves the image stitching efficiency, with an average acceleration rate of about 2.74 times, providing a reliable and efficient reference solution for engineering implementation.

[0020] Specifically, the coordinate matrix P of the image in step (2) is obtained based on the similarity of the overlapping areas of adjacent images. The coordinate matrix P includes ROW×COL groups of data, each group of data represents the coordinate information of the first pixel in each image, the horizontal coordinate is marked as x, the vertical coordinate is marked as y, and the coordinate of the first pixel of the image in row row and col column is marked as P(row,col)=(x row_col ,y row_col ).

[0021] Specifically, the specific steps of calculating the relative position offset matrix M between each image and the adjacent upper image and the adjacent left image in step (3) are:

[0022] Step (3-1): Define the relative position offset matrix M to contain ROW×COL groups of data, each group of data contains 4 data: the X-direction misalignment relationship between the current image and the adjacent left image, recorded as M(row,col) xLR ; The Y-direction misalignment relationship between the current image and the adjacent left image, denoted as M(row,col) yLR ; The X-direction misalignment relationship between the current image and the adjacent upper image, denoted as M(row,col) xUD ; The Y-direction misalignment relationship between the current image and the adjacent upper image, denoted as M(row,col) yUD , initialize all the data of the relative position offset matrix M to the same integer value, denoted as error, and use M(row,col) to represent the data of the relative position offset matrix M at the rowth row and the colth column;

[0023] Step (3-2): According to the coordinate matrix P, calculate the data of the relative position offset matrix M row by row:

[0024]

[0025] In the formula, h and w are the height and width of each image, which means that the Y direction contains h pixels and the X direction contains w pixels; row_col is the starting horizontal coordinate of the image at row row and col column, x row_col-1 The starting horizontal coordinate of the image at row row and col-1 column is row_col and (x row_col-1 +w) to obtain the X-direction offset M(row,col) between the current image and the adjacent left image. xLR ;y row_col Represents the starting ordinate of the image at row row and column col, y row_col-1 Represents the starting ordinate of the image at row row and col-1 column, through y row_col With y row_col-1The difference between the two is used to obtain the Y offset M (row, col) between the current image and the adjacent left image. yLR ;x row-1_col Represents the starting horizontal coordinate of the image at row-1 and column col, through x row_col With x row-1_col The difference between the current image and the adjacent image is obtained by the X-axis offset M (row, col) xUD ;y row-1_col Represents the starting ordinate of the image at row-1 and column col, through y row_col and (y row-1_col +h) to obtain the Y offset M(row,col) between the current image and the adjacent image. yUD ; If the adjacent left image or adjacent upper image of the current image does not exist, the relevant parameters in the relative position offset matrix M are kept as initial values;

[0026] Step (3-3): Calculate all data in the matrix M to obtain the complete relative position offset matrix M.

[0027] Furthermore, the specific method of obtaining the first part of the image in step (4) is:

[0028] Step (4-4-1): Use QtConcurrent::run method to start multi-threaded task and execute the image stitching task of non-overlapping areas of COL images in row;

[0029] Step (4-4-2): Read the image at row and column, denoted as Img row_col , stored in the computer's memory;

[0030] Step (4-4-3): Get the starting coordinates of the image at row row and column col, recorded as (x start ,y start ):

[0031]

[0032] Among them, x start Represents the starting horizontal coordinate of the image at row row and column col, y start Represents the starting ordinate of the image at row row and column col, x row_col ,y row_col All come from the coordinate matrix P;

[0033] Step (4-4-4): Get the current image Img according to the relative position offset matrix M row_col With the adjacent left image Img row_col-1 X-axis offset value M(row,col)xLR , get the current image Img row_col With the adjacent image Img row-1_col Y offset value M(row,col) yUD ;

[0034] Step (4-4-5): Convert the current image Img row_col The non-overlapping area is assigned to the original large image Img Whole :

[0035] (a) If row = 1 and col = 1, do the following:

[0036]

[0037] Among them, Img Whole (x k ,y k ) represents the original large image Img Whole The coordinate (x k ,y k ) and there is x k ∈[1,h Whole ]、y k ∈[1,w Whole ],Img row_col (x,y) represents the current image Img row_col The pixel value at coordinate (x, y) in the image, x1 and x2 represent the current image Img. row_col The starting and ending horizontal coordinates of the X direction, y1 and y2 represent the current image Img row_col The starting and ending ordinates of the Y direction, with x∈[x1,x2], y∈[y1,y2];

[0038] (b) If row = 1 and col > 1, do the following:

[0039]

[0040] (c) If row>1 and col=1, do the following:

[0041]

[0042] (d) If row>1 and col>1, do the following:

[0043]

[0044] Step (4-4-6): Take the row COL image Img row_col The non-overlapping area is assigned to the original large image Img Whole, the first part of the image is obtained.

[0045] Further, the specific method for obtaining the second part of the image in step (4) is as follows:

[0046] Step (4-5-1): Use the QtConcurrent::run method to execute the image fusion task for the overlapping part of the first type of two images in the COL images of the row-th row. The overlapping part of the first type of two images refers to the part that does not affect the fusion effect of the four-image overlapping area;

[0047] Step (4-5-2): Obtain the starting coordinates of the image in the row-th row and col-th column, denoted as (x start , y start );

[0048] Step (4-5-3): Perform image fusion on the area where there is an overlap between two images in the current image Img row_col , and update the result to the original large image Img Whole :

[0049] (a) If row = 1 and col > 1, perform image fusion on the overlapping area of two images in the X direction, and perform the following operations:

[0050]

[0051] Among them, p represents the weight ratio of image fusion, and v update represents the value updated to the original large image Img Whole at the coordinate (x k , y k );

[0052] (b) If row > 1, first perform image fusion on the overlapping area of two images in the Y direction for all images in the row-th row, and perform the following operations:

[0053] First, set x1 and x2. If col = 1, let x1 = 1, otherwise, let x1 = |M(row, col) xLR | + 1; if col < COL, let x2 = w - |M(row, col + 1) xLR |, otherwise, let x2 = w;

[0054] Then, set y1 and y2. If col < COL and satisfies y row_col < y row_col+1 , let y1 = y row_col+1 - y row_col , otherwise, let y1 = 1; if col < COL and satisfies [y row_col + |M(row, col) yLR |] < [yrow_col+1 +|M(row, col + 1) yLR |], let y2 = |M(row, col) yLR |-{[y row_col+1 +|M(row, col + 1) yLR |]-[y row_col +|M(row, col) yLR |]}, otherwise, let y2 = h;

[0055] Then, based on y1 and y2, obtain the height difference of the overlapping region of the two images in the Y direction, denoted as h yDelta ;

[0056] h yDelta = y2 - y1 + 1

[0057] Finally, perform the following image fusion for the overlapping region of the two images in the Y direction:

[0058]

[0059] After completion, if col < COL and it meets [y row_col +|M(row, col) yUD |] > [y row_col+1 +|M(row, col + 1) yUD |], then directly overwrite the corresponding region of the original large image Img row_col in the following way: Whole of the corresponding region:

[0060]

[0061] Step (4 - 5 - 4): Use the QtConcurrent::run method to execute the image fusion task for the second type of overlapping part of the COL images in the row-th row, where the second type of overlapping part of the two images refers to the part that affects the fusion effect of the four-image overlapping region;

[0062] Step (4 - 5 - 5): Obtain the starting coordinates of the image in the row-th row and col-th column, denoted as (x start , y start );

[0063] Step (4 - 5 - 6): If row > 1 and col > 1, perform image fusion for the overlapping region of the two images in the X direction for all images in the row-th row, and perform the following operations:

[0064] First, set x1 and x2, let x1 = 1; let x2 = |M(row, col) xLR |;

[0065] Then, set y1 and y2, if yrow_col-1 +|M(row,col-1) yUD |]<[y row_col +|M(row,col) yUD |], let y1 = |M(row,col) yUD |-{[y row_col +|M(row,col) yUD |-[y row_col-1 +|M(row,col-1) yUD |]}, otherwise, let y1=|M(row,col) yUD |;y2=h;

[0066] Then, based on x1 and x2, we get the height difference of the overlapping area of ​​the two images in the X direction, which is recorded as w xDelta :

[0067] w xDelta =x2-x1+1

[0068] Finally, perform image fusion of the overlapping area of ​​the two images in the X direction:

[0069]

[0070] Step (4-5-7): Complete the image fusion of the overlapping area of ​​the two images in the row row COL images to obtain the second part of the image.

[0071] Furthermore, the specific method for obtaining the third part of the image in step (4) is:

[0072] Step (4-6-1): Use QtConcurrent::run method to perform the image fusion task of the four overlapping areas of the row-th row image that is not the first column; take the current image as the position reference object, record the upper right corner of the adjacent left image as the first overlapping area, record the upper left corner of the current image as the second overlapping area, and record the original large image Img Whole The local overlapping area of ​​the four images in the filled image is recorded as the third overlapping area;

[0073] Step (4-6-2): Obtain a local image of the first overlapping area, denoted as Img1 row_col-1 ;

[0074] First, set x1 and x2, let x1 = w-|M(row,col) xLR |+1; let x2=w;

[0075] Then, set y1, y2, if y row_col-1 <y row_col , let y1 = y row_col -y row_col-1+1, otherwise, let y1=1; if [y row_col-1 +|M(row,col-1) yUD |]>[y row_col +|M(row,col) yUD |], let y2 = |M(row,col) yUD |-{[y row_col-1 +|M(row,col-1) yUD |]-[y row_col +|M(row,col) yUD |]}+1, otherwise, let y2=|M(row,col-1) yUD |+1;

[0076] Finally, perform the following image copy operation to obtain the local image Img1 of the first overlapping area: row_col-1 :

[0077]

[0078] Among them, Img1 row_col-1 (x k ,y k ) represents the local image Img1 of the first overlapping area row_col-1 The coordinate (x k ,y k ) pixel value, Img row_col-1 (x k ,y k ) represents the local image Img of the first overlapping area row_col-1 The pixel value at coordinate (x, y) in ;

[0079] Step (4-6-3): Obtain a local image of the second overlapping area, denoted as Img2 row_col ;

[0080] First, set x1 and x2, let x1 = 1; let x2 = |M(row,col) xLR |;

[0081] Then, set y1, y2, if y row_col-1 >y row_col , let y1 = y row_col-1 -y row_col +1, otherwise, let y1=1; if [y row_col-1 +|M(row,col-1) yUD |]<[y row_col +|M(row,col) yUD |], let y2 = |M(row,col) yUD |-{[yrow_col +|M(row,col) yUD |]-[y row_col-1 +|M(row,col-1) yUD |]}+1, otherwise, let y2=|M(row,col) yUD |+1;

[0082] Finally, perform the following image copy operation to obtain the local image Img2 of the second overlapping area: row_col :

[0083]

[0084] Among them, Img2 row_col (x k ,y k ) represents the local image Img2 of the second overlapping area row_col The coordinate (x k ,y k ) pixel value, Img row_col (x k ,y k ) represents the local image Img of the second overlapping area row_col The pixel value at coordinate (x, y) in ;

[0085] Step (4-6-4): The local image Img1 of the second overlapping area row_col-1 The local image Img2 of the second overlapping area row_col Perform linear image fusion along the X direction to obtain a temporary fused image, denoted as Img temp_row_col , the width is denoted as w temp_row_col , the height is denoted as h temp_row_col , the local image Img1 of the first overlapping area row_col-1 , the local image Img2 of the second overlapping area row_col And the temporary fusion image Img temp_row_col The sizes are all the same;

[0086]

[0087] Step (4-6-5): Temporary fusion image Img temp_row_col With the original large image Img Whole The corresponding overlapping areas are linearly merged along the Y direction:

[0088] First, set x1 and x2, and let x1 = x row_col ; Let x2=x1+w temp_row_col -1;

[0089] Then, set y1, y2, if xrow_col-1 >x row_col , let x1=x row_col-1 , otherwise, let x1 = x row_col ; Let x2=x1+w temp_row_col -1;

[0090]

[0091] Step (4-6-6): Complete the image fusion of the overlapping area of ​​the four images in the row-th row, and obtain the third part of the image. BRIEF DESCRIPTION OF THE DRAWINGS

[0092] Figure 1 It is a general flow chart of the stitching method based on improved linear fusion microscopic images of the present invention;

[0093] Figure 2 A flowchart of a specific method for stitching microscopic images of the present invention;

[0094] Figure 3 A schematic diagram of a scanning and photographing path of a microscopic image stitching method of the present invention;

[0095] Figure 4 It is a schematic diagram of the relative positions of four adjacent figures according to an embodiment of the present invention;

[0096] Figure 5 A schematic diagram of a real sample of an embodiment of the present invention (8 rows×7 columns);

[0097] Figure 6 A schematic diagram of a local area of ​​a real sample according to an embodiment of the present invention (2 rows×2 columns);

[0098] Figure 7 This is a schematic diagram of the local area of ​​image fusion before improvement;

[0099] Figure 8 It is a schematic diagram of a local area of ​​image fusion after the improvement of the microscopic image stitching method of the present invention. DETAILED DESCRIPTION

[0100] The present invention is further described in detail below with reference to the accompanying drawings.

[0101] Embodiment: The process of the microscopic image stitching method based on improved linear fusion of the present invention is as follows: Figure 1 As shown, Figure 2 FIG. 4 is a flow chart of a specific method for image stitching of the present invention, which will be specifically introduced in the following embodiments.

[0102] In general microscopic sample image stitching, scanning, stitching calculation and image fusion are performed successively. The scanning and image taking path of the present invention adopts a "snake-like" method, such as Figure 3 As shown, this method can improve the image acquisition efficiency compared to the "return-type" scanning and imaging path, and is commonly seen in popular slice scanning or microscope scanning equipment.

[0103] like Figure 4 This is a schematic diagram of the relative positions of the four adjacent images of the present invention. Due to factors such as the movement accuracy of the electric platform, improper camera assembly, and jitter in scanning and shooting, there is a random misalignment relationship between the adjacent images taken, and the effect of the splicing calculation directly affects the puzzle effect. This patent mainly focuses on eliminating splicing marks, so the present invention does not explain in detail how to calculate the splicing calculation. The patent of this invention focuses on solving the problem of image splicing marks caused by uneven lighting brightness. Abnormal splicing marks are common in Figure 4 The four-image overlapping area (the area marked with “horizontal stripes”) and the two-image overlapping area (the area marked with “diagonal stripes”).

[0104] like Figure 5 Schematic diagram of a real sample of the present invention, which includes 8 rows and 7 columns of pictures. The resolution of each picture is 2048×2048, which is a 16-bit single-channel image (non-color image).

[0105] For the above real samples, according to Figure 1 and Figure 2 , the implementation method of the present invention comprises the following steps:

[0106] Step (1): Set the total number of scanned images to ROW and COL. In this embodiment, ROW=8, COL=7. Set the position of each image to the number "row number_column number". The row number is recorded as row, row∈[1,8], and the column number is recorded as col, col∈[1,7].

[0107] Step (2): Based on the result of the stitching calculation, the coordinate matrix of each image is obtained, which is denoted as P. The coordinate matrix P can be obtained based on the similarity of the overlapping areas of adjacent images. For details, please refer to patent CN111626933B. P contains 8×7 groups of data in total, each group of data represents the coordinate information of the first pixel of each image (the first pixel is located in the first row and the first column of the image). The horizontal coordinate is marked as x and the vertical coordinate is marked as y. Then, the coordinate of the first pixel of the image in row row and col column is marked as P(row,col)=(x row_col ,y row_col ), for the sake of simplicity, the "coordinate of the first pixel of the image" is referred to as the "starting coordinate of the image", the "horizontal coordinate of the first pixel of the image" is referred to as the "starting horizontal coordinate of the image", and the "vertical coordinate of the first pixel of the image" is referred to as the "starting vertical coordinate of the image";

[0108] Step (3): Calculate the relative position offset matrix between each image and the adjacent upper image and the adjacent left image, denoted as M, specifically:

[0109] Step (3-1): Define the relative position offset matrix M to contain 8×7 groups of data, each group of data contains 4 data: the X-direction misalignment relationship between the current image and the adjacent left image, recorded as M(row,col) xLR ; The Y-direction misalignment relationship between the current image and the adjacent left image, denoted as M(row,col) yLR ; The X-direction misalignment relationship between the current image and the adjacent upper image, denoted as M(row,col) xUD ; The Y-direction misalignment relationship between the current image and the adjacent upper image, denoted as M(row,col) yUD , initialize all the data of the relative position offset matrix M to the same integer value, denoted as error, and use M(row,col) to represent the data of the relative position offset matrix M at the rowth row and the colth column;

[0110] Step (3-2): According to the coordinate matrix P, calculate the data of the relative position offset matrix M row by row:

[0111]

[0112] Wherein, h and w are the height and width of each image, indicating that the Y direction contains h pixels and the X direction contains w pixels. In this embodiment, h = 2048, w = 2048; x row_col is the starting horizontal coordinate of the image at row row and col column, x row_col-1 The starting horizontal coordinate of the image at row row and col-1 column is row_col With (x row_col-1 +w) to obtain the X-direction offset M(row,col) between the current image and the adjacent left image. xLR ;y row_col Represents the starting ordinate of the image at row row and column col, y row_col-1 Represents the starting ordinate of the image at row row and col-1 column, through y row_col With y row_col-1 The difference between the two is used to obtain the Y offset M (row, col) between the current image and the adjacent left image. yLR ;x row-1_col Represents the starting horizontal coordinate of the image at row-1 and column col, through x row_col With x row-1_col The difference between the current image and the adjacent image is obtained by the X-axis offset M (row, col) xUD ;y row-1_colRepresents the starting ordinate of the image at row-1 and column col, through y row_col and (y row-1_col +h) to obtain the Y offset M(row,col) between the current image and the adjacent image. yUD ; If the adjacent left image or the adjacent upper image of the current image does not exist, the relevant data in the relative position offset matrix M is kept as the initial data;

[0113] Step (3-3): Calculate all data in the matrix M to obtain a complete relative position offset matrix M;

[0114] Step (4): Figure 2 As shown, multi-threading technology is used to achieve image fusion and complete the stitching of microscopic images. The specific method is as follows:

[0115] Step (4-1): Create a blank image and use it as the current image, denoted as Img Whole , initialize the row number, set row = 1, Img Whole The size depends on the coordinate matrix P, the image height h and the image width w. Initially, the blank image Img Whole The pixel points are initially set to 0 value;

[0116]

[0117] Among them, w Whole and h Whole Represents blank large image Img Whole The width and height of max(x row_col )、max(x row_col ) represent the maximum and minimum horizontal coordinates in the coordinate matrix P respectively; max(y row_col )、max(y row_col ) represent the maximum and minimum vertical coordinates in the coordinate matrix P respectively.

[0118] Step (4-2): Determine whether the row number row meets the condition "row≤8". If so, execute step (4-3); otherwise, execute step (4-8);

[0119] Step (4-3): Use the current large image as the original large image and add 1 to the row number row;

[0120] Step (4-4): According to the coordinate matrix P and the relative position offset matrix M, the images of the non-overlapping areas of the COL images in the rowth row are spliced ​​to obtain the first partial image, specifically:

[0121] Step (4-4-1): Start the first round of multi-threaded tasks, use the QtConcurrent::run method to start the multi-threaded tasks, and concurrently execute the image stitching task of the non-overlapping areas of the 7 images in row; the QtConcurrent::run method is a multi-threaded technology of the Qt software development platform, and the official website can be found at https: / / doc.qt.io / qt-6 / qtconcurrentrun.html;

[0122] Step (4-4-2): Read the image at row and column, denoted as Img row_col , stored in the computer's memory;

[0123] Step (4-4-3): Get the starting coordinates of the image at row row and column col, recorded as (x start ,y start ):

[0124]

[0125] Among them, x start Represents the starting horizontal coordinate of the image at row row and column col, y start Represents the starting ordinate of the image at row row and column col, x row_col ,y row_col All come from the coordinate matrix P;

[0126] Step (4-4-4): Get the current image Img according to the relative position offset matrix M row_col With the adjacent left image Img row_col-1 X-axis offset value M(row,col) xLR , get the current image Img row_col With the adjacent image Img row-1_col Y offset value M(row,col) yUD ;

[0127] Step (4-4-5): Convert the current image Img row_col The non-overlapping area is assigned to the original large image Img Whole There are four modes:

[0128] (a) If "row = 1 and col = 1", do the following:

[0129]

[0130] Among them, Img Whole (x k ,y k ) represents the original large image Img WholeThe pixel value at the coordinate (x k , y k ), where x k ∈ [1, h Whole , y k ∈ [1, w Whole . In this embodiment, h Whole = 13699 and w Whole = 11925. All subsequent operations comply with this boundary condition and will not be elaborated further; Img row_col (x, y) represents the pixel value at the coordinate (x, y) in the current image Img row_col . x1 and x2 respectively represent the starting abscissa and ending abscissa in the X direction in the current image Img row_col . y1 and y2 respectively represent the starting ordinate and ending ordinate in the Y direction in the current image Img row_col , and x ∈ [x1, x2], y ∈ [y1, y2];

[0131] (b) If "row = 1 and col > 1", perform the following operations:

[0132]

[0133] (c) If "row > 1 and col = 1", perform the following operations:

[0134]

[0135] (d) If "row > 1 and col > 1", perform the following operations:

[0136]

[0137] Step (4-4-6): Assign the non-overlapping regions of the COL images Img row_col in the row-th row to the original large image Img Whole to obtain the first part of the image, providing a data basis for subsequent image fusion, and retaining all the images Img row_col in the row-th row in memory;

[0138] Step (4-5): According to the coordinate matrix P and the relative position offset matrix M, perform image fusion on the two-image overlapping regions of the 7 images in the row-th row to obtain the second part of the image. The specific method is as follows:

[0139] Step (4-5-1): Start the second round of multi-threaded tasks, and use the QtConcurrent::run method to concurrently execute the image fusion tasks for the first type of two-image overlapping parts in the 7 images in the row-th row that do not affect the fusion effect of the four-image overlapping regions;

[0140] Step (4-5-2): Get the starting coordinates of the image at row row and column col, recorded as (x start ,y start );

[0141] Step (4-5-3): For the current image Img row_col Image fusion is performed in the area where the two images overlap, and the result is updated to the original large image Img Whole , there are two modes:

[0142] (a) If row = 1 and col > 1, perform image fusion of the overlapping area of ​​the two images in the X direction and perform the following operations:

[0143]

[0144] Among them, p represents the weight ratio of image fusion, v update Represents update to large image Img Whole The coordinate (x k ,y k )

[0145] (b) If "row>1", all images in row 1 are first fused in the Y-direction in the overlapping area of ​​the two images, and the following operations are performed:

[0146] First, set x1 and x2. If "col=1", set x1=1. Otherwise, set x1=|M(row,col) xLR |+1; if "col<7", let x2=w-|M(row,col+1) xLR |, otherwise, let x2 = w;

[0147] Then, set y1 and y2. If "col<7" and "y row_col <y row_col+1 ”, let y1=y row_col+1 -y row_col , otherwise, let y1 = 1; if "col < 7" and meets the "[y row_col +|M(row,col) yLR |]<[y row_col+1 +|M(row,col+1) yLR |]", let y2 = |M(row,col) yLR |-{[y row_col+1 +|M(row,col+1) yLR |]-[y row_col +|M(row,col) yLR |]}, otherwise, let y2 = h;

[0148] Then, based on y1 and y2, we get the height difference of the overlapping area of ​​the two images in the Y direction, which is recorded as h. yDelta ;

[0149] h yDelta =y2-y1+1

[0150] Finally, perform the following image fusion of the overlapping area of ​​the two images in the Y direction:

[0151]

[0152] After the end, if "col<7" and meets the "[y row_col +|M(row,col) yUD |]>[y row_col+1 +|M(row,col+1) yUD |]", then the current image Img row_col Directly overwrite the original large image Img in the following way Whole The corresponding area:

[0153]

[0154] Step (4-5-4): Start the third round of multi-threaded tasks, and use the QtConcurrent::run method to execute the image fusion task of the second type of overlapping parts of two images in the row, which will affect the fusion effect of the overlapping area of ​​the four images;

[0155] Step (4-5-5): Get the starting coordinates of the image at row row and column col, recorded as (x start ,y start ).

[0156] Step (4-5-6): If "row>1 and col>1", all images in row 1 are merged in the X-direction in the overlapping area of ​​the two images. Perform the following operations:

[0157] First, set x1 and x2, let x1 = 1; let x2 = |M(row,col) xLR |;

[0158] Then, set y1 and y2. If "[y row_col-1 +|M(row,col-1) yUD |]<[y row_col +|M(row,col) yUD |]", let y1=|M(row,col) yUD |-{[y row_col +|M(row,col) yUD |-[y row_col-1+|M(row,col-1) yUD |]}, otherwise, let y1=|M(row,col) yUD |;y2=h;

[0159] Then, set y1 and y2. If "[y row_col-1 +|M(row,col-1) yUD |]<[y row_col +|M(row,col) yUD |]", let y1=|M(row,col) yUD |-{[y row_col +|M(row,col) yUD |-[y row_col-1 +|M(row,col-1) yUD |]}, otherwise, let y1=|M(row,col) yUD |;y2=h;

[0160] Then, based on x1 and x2, we get the height difference of the overlapping area of ​​the two images in the X direction, which is recorded as w xDelta :

[0161] w xDelta =x2-x1+1

[0162] Finally, perform image fusion of the overlapping area of ​​the two images in the X direction:

[0163]

[0164] Step (4-5-7): Complete the image fusion of the overlapping area of ​​the two images of the 7 images in row to obtain the second part of the image;

[0165] Step (4-6): According to the coordinate matrix P and the relative position offset matrix M, image fusion is performed on the overlapping area of ​​the four images in the row row COL images to obtain the third part of the image. The specific method is:

[0166] Step (4-6-1): Start the fourth round of multi-threaded tasks, and use the QtConcurrent::run method to concurrently execute the image fusion task of the four-image overlapping area of ​​the row-th row image that is not the first column; the four-image overlapping area refers to the common overlapping area of ​​four adjacent images, which usually appears in images that are not in the first row or column. In order to illustrate the fusion method of the four-image overlapping area, it is necessary to clarify the specific scope of image fusion. Take the current image as the position reference object, record the upper right corner of the adjacent left image as the first overlapping area, record the upper left corner of the current image as the second overlapping area, and record the original large image Img Whole The local overlapping area of ​​the four images in the filled image is recorded as the third overlapping area;

[0167] Step (4-6-2): Obtain a local image of the first overlapping area, denoted as Img1 row_col-1 ;

[0168] First, set x1 and x2, let x1 = w-|M(row,col) xLR |+1; let x2=w;

[0169] Then, set y1, y2, if y r ow_col-1 <y row_col , let y1 = y row_col -y row_col-1 +1, otherwise, let y1=1; if [y row_col-1 +|M(row,col-1) yUD |]>[y row_col +|M(row,col) yUD |], let y2 = |M(row,col) yUD |-{[y row_col-1 +|M(row,col-1) yUD |]-[y row_col +|M(row,col) yUD |]}+1, otherwise, let y2=|M(row,col-1) yUD |+1;

[0170] Finally, perform the following image copy operation to obtain the local image Img1 of the first overlapping area: row_col-1 :

[0171]

[0172] Among them, Img1 row_col-1 (x k ,y k ) represents Img1 row_col-1 The coordinate (x k ,y k ) pixel value, Img row_col-1 (x k ,y k ) represents the local image Img of the first overlapping area row_col-1 The pixel value at coordinate (x, y) in ;

[0173] Step (4-6-3): Obtain a local image of the second overlapping area, denoted as Img2 row_col ;

[0174] First, set x1 and x2, let x1 = 1; let x2 = |M(row,col) xLR |;

[0175] Then, set y1, y2, if y row_col-1 >y row_col , let y1 = y row_col-1 -y row_col +1, otherwise, let y1=1; if [y row_col-1 +|M(row,col-1) yUD |]<[y row_col +|M(row,col) yUD |], let y2 = |M(row,col) yUD |-{[y row_col +|M(row,col) yUD |]-[y row_col-1 +|M(row,col-1) yUD |]}+1, otherwise, let y2=|M(row,col) yUD |+1;

[0176] Finally, perform the following image copy operation to obtain the local image Img2 of the second overlapping area: row_col :

[0177]

[0178] Among them, Img2 row_col (x k ,y k ) represents the local image Img2 of the second overlapping area row_col The coordinate (x k ,y k ) pixel value, Img row_col (x k ,y k ) represents the local image Img of the second overlapping area row_col The pixel value at coordinate (x, y) in ;

[0179] Step (4-6-4): The local image Img1 of the first overlapping area row_col-1 The local image Img2 of the second overlapping area row_col Perform linear image fusion along the X direction to obtain a temporary fused image, denoted as Img temp_row_col , the width is denoted as w temp_row_col , the height is denoted as h temp_row_col , the local image Img1 of the first overlapping area row_col-1 , the local image Img2 of the second overlapping area row_col And the temporary fusion image Img temp_row_col The sizes are all the same;

[0180]

[0181] Step (4-6-5): Temporary fusion image Img temp_row_col With the original large image Img Whole The corresponding overlapping areas of the four images are linearly fused along the Y direction, which is equivalent to the bilinear fusion of the overlapping areas of the four images:

[0182] First, set x1 and x2, and let x1 = x row_col ; Let x2=x1+w temp_row_col -1;

[0183] Then, set y1, y2, if x row_col-1 >x row_col , let x1=x row_col-1 , otherwise, let x1 = x row_col ; Let x2=x1+w temp_row_col -1;

[0184]

[0185] Step (4-6-6): Complete the image fusion of the overlapping area of ​​the four images of the 7 images in row to obtain the third part of the image;

[0186] Step (4-7): assign the first partial image, the second partial image and the third partial image to the original large image to obtain the current large image, and return to step (4-2);

[0187] Step (4-8): Use the current large image as the stitched microscopic image to obtain a complete, naturally integrated stitched large image, namely Img Whole .

[0188] The method of the present invention is experimentally verified as follows.

[0189] 1. Experimental environment

[0190] (1) Processor: AMD Rayzen 95900HS, 8 cores;

[0191] (2) Memory: 32 GB;

[0192] (3) Hard disk: 2T solid state hard disk;

[0193] (4) Operating system: Windows 10, 64-bit;

[0194] 2. Qualitative Experiment

[0195] This experiment mainly compares the image fusion effects before and after improvement. The original image for image fusion is selected Figure 5 The local area of ​​the sample, such as Figure 6 As shown, the number of images in the local area is 2 rows × 2 columns. Figure 7This is a schematic diagram of the local area of ​​image fusion before improvement. Figure 8 This is a schematic diagram of the local area of ​​image fusion after the improved splicing method of the present invention is used. Figure 7 On the right side, the image fusion area of ​​interest is magnified, which is the image fusion result of the overlapping area of ​​the four images. Figure 8 The right side of the figure also shows the enlarged image fusion result of the overlapping area of ​​the four images of interest. Figure 7 There are two obvious horizontal and vertical stitching marks in the overlapping area of ​​the four images. Figure 8 There is no obvious stitching trace in the overlapping area of ​​the four images, and the pixel brightness transition is natural.

[0196] The results show that the present invention eliminates the image mosaic problem caused by uneven illumination brightness and improves the shortcomings of the linear fusion method.

[0197] 3. Quantitative Experiment

[0198] In order to improve the efficiency of image stitching, the present invention adopts multi-threading technology in the image stitching process. Table 1 is a comparison table of the image stitching time before and after multi-threading acceleration. The stitching time here specifically refers to the process from step (4-1) to step (4-7), and does not include how to obtain the coordinate matrix P, the relative position offset matrix M, and the link of saving the large image.

[0199] Table 1 records the time consumption of 5 different types of samples. The results show that after multi-threaded acceleration, the average acceleration rate of image stitching is about 2.74 times, which significantly improves the stitching efficiency and provides the possibility for the transformation from theory to engineering implementation.

[0200] Table 1 Comparison of image stitching time before and after multithreading acceleration

[0201]

Claims

1. A microscopic image stitching method based on improved linear fusion, comprising scanning and taking pictures, stitching calculation and image fusion, characterized in that The image fusion comprises the following steps: Step (1): Set the total number of scanned images to ROW and COL, set the position of each image to the number "row number_column number", record the row number as row, row∈[1,ROW], and record the column number as col, col∈[1,COL]; Step (2): According to the result of the splicing calculation, the coordinate matrix of each image is obtained, denoted as P; Step (3): Calculate the relative position offset matrix between each image and the adjacent upper image and the adjacent left image, denoted as M; Step (4): Use multi-threading technology to achieve image fusion and complete the stitching of microscopic images. The specific method is as follows: Step (4-1): Create a blank image and use it as the current image, initialize the row number, and set row = 1; Step (4-2): Determine whether the row number row meets the condition row≤ROW. If so, execute step (4-3); Otherwise, execute steps (4-8); Step (4-3): Use the current large image as the original large image and add 1 to the row number row; Step (4-4): according to the coordinate matrix P and the relative position offset matrix M, stitch the images of the non-overlapping areas of the COL images in the row th row to obtain the first partial image; Step (4-5): According to the coordinate matrix P and the relative position offset matrix M, image fusion is performed on the overlapping area of ​​the two images in the row row COL images to obtain the second part of the image; Step (4-6): According to the coordinate matrix P and the relative position offset matrix M, image fusion is performed on the overlapping area of ​​the four images in the row-th row COL images to obtain the third partial image; Step (4-7): assigning the first partial image, the second partial image and the third partial image to the original large image to obtain the current large image, and returning to step (4-2); Step (4-8): Use the current large image as the stitched microscopic image.

2. A method for stitching microscopic images based on improved linear fusion as claimed in claim 1, characterized in that The coordinate matrix P of the image in step (2) is obtained based on the similarity of the overlapping areas of adjacent images. The coordinate matrix P includes ROW×COL groups of data, each group of data represents the coordinate information of the first pixel in each image, the horizontal coordinate is marked as x, the vertical coordinate is marked as y, and the coordinate of the first pixel of the image in row row and col column is marked as P(row,col)=(x row_col ,y row_col ).

3. A method for stitching microscopic images based on improved linear fusion as claimed in claim 2, characterized in that The specific steps of calculating the relative position offset matrix M of each image and the adjacent upper image and adjacent left image in step (3) are as follows: Step (3-1): Define the relative position offset matrix M to contain ROW×COL groups of data, each group of data contains 4 data: the X-direction misalignment relationship between the current image and the adjacent left image, recorded as M(row,col) xLR ; The Y-direction misalignment relationship between the current image and the adjacent left image, denoted as M(row,col) yLR ; The X-direction misalignment relationship between the current image and the adjacent upper image, denoted as M(row,col) xUD ; The Y-direction misalignment relationship between the current image and the adjacent upper image, denoted as M(row,col) yUD , initialize all the data of the relative position offset matrix M to the same integer value, denoted as error, and use M(row,col) to represent the data of the relative position offset matrix M at the rowth row and the colth column; Step (3-2): According to the coordinate matrix P, calculate the data of the relative position offset matrix M row by row: In the formula, h and w are the height and width of each image, which means that the Y direction contains h pixels and the X direction contains w pixels; row_col is the starting horizontal coordinate of the image at row row and col column, x row_col-1 The starting horizontal coordinate of the image at row row and col-1 column is row_col and (x row_col-1 +w) to obtain the X-direction offset M(row,col) between the current image and the adjacent left image. xLR ;y row_col Represents the starting ordinate of the image at row row and column col, y row_col-1 Represents the starting ordinate of the image at row row and col-1 column, through y row_col With y row_col-1 The difference between the two is used to obtain the Y offset M (row, col) between the current image and the adjacent left image. yLR ;x row-1_col Represents the starting horizontal coordinate of the image at row-1 and column col, through x row_col With x row-1_col The difference between the current image and the adjacent image is obtained by the X-axis offset M (row, col) xUD ;y row-1_col Represents the starting ordinate of the image at row-1 and column col, through y row_col and (y row-1_col +h) to obtain the Y offset M(row,col) between the current image and the adjacent image. yUD ; If the adjacent left image or the adjacent upper image of the current image does not exist, the relevant data in the relative position offset matrix M is kept as the initial data; Step (3-3): Calculate all data in the matrix M to obtain the complete relative position offset matrix M.

4. A method for stitching microscopic images based on improved linear fusion as claimed in claim 3, characterized in that The specific method for obtaining the first part of the image in step (4) is: Step (4-4-1): Use QtConcurrent::run method to start multi-threaded task and execute the image stitching task of non-overlapping areas of COL images in row; Step (4-4-2): Read the image at row and column, denoted as Img row_col , stored in the computer's memory; Step (4-4-3): Get the starting coordinates of the image at row row and column col, recorded as (x start ,y start ): Among them, x start Represents the starting horizontal coordinate of the image at row row and column col, y start Represents the starting ordinate of the image at row row and column col, x r ow_col,y r ow_col all come from the coordinate matrix P; Step (4-4-4): Get the current image Img according to the relative position offset matrix M row_col With the adjacent left image Img row_col-1 X-axis offset value M(row,col) xLR , get the current image Img r ow_col and the adjacent image above Img r ow-1_col's Y offset value M(row,col) yUD ; Step (4-4-5): Convert the current image Img r The non-overlapping area of ​​ow_col is assigned to the original large image Img Wh ole: (a) If row = 1 and col = 1, do the following: x∈[x1,x2],y∈[y1,y2]where Img Whole (x k ,y k ) represents the original large image Img Whole The coordinate (x k ,y k ) and there is x k ∈[1,h Whole ]、y k ∈[1,w Whole ], Img row_col (x,y) represents the current image Img row_col The pixel value at coordinate (x, y) in the image, x1 and x2 represent the current image Img. row_col The starting and ending horizontal coordinates of the X direction, y1 and y2 represent the current image Img row_col The starting and ending ordinates of the Y direction, with x∈[x1,x2], y∈[y1,y2]; (b) If row = 1 and col > 1, do the following: <h2 style=";text-align:left;direction:ltr"> <h2 style=";text-align:left;direction:ltr"> x∈[x1,x2],y∈[y1,y2] (c) If row>1 and col=1, do the following: <h2 style=";text-align:left;direction:ltr"> <h2 style=";text-align:left;direction:ltr"> x∈[x1,x2],y∈[y1,y2] (d) If row>1 and col>1, do the following: x∈[x1,x2],y∈[y1,y2] Step (4-4-6): convert row COL images Img row_col The non-overlapping area is assigned to the original large image Img Whole , get the first part of the image.

5. A method for stitching microscopic images based on improved linear fusion as claimed in claim 3, characterized in that The specific method for obtaining the second part of the image in step (4) is: Step (4-5-1): Use QtConcurrent::run method to execute the image fusion task of the first type of overlapping part of two images in row COL images, where the first type of overlapping part of two images refers to the part that does not affect the fusion effect of the overlapping area of ​​four images; Step (4-5-2): Get the starting coordinates of the image at row row and column col, recorded as (x start ,y start ); Step (4-5-3): For the current image Img row_col Image fusion is performed in the area where the two images overlap, and the result is updated to the original large image Img Whole : (a) If row = 1 and col > 1, perform image fusion in the overlapping area of ​​the two images in the X direction and perform the following operations: x∈[x1,x2],y∈[y1,y2], where p represents the weight of image fusion, v update Represents update to the original large image Img Whole The coordinate (x k ,y k ) (b) If row>1, all images in row are first fused in the Y-direction in the overlapping area of ​​the two images, and the following operations are performed: First, set x1 and x2. If col = 1, let x1 = 1, otherwise, let x1 = |M(row,col) xLR | + 1; if col < COL, let x2 = w - |M(row,col+1) xLR |, otherwise, let x2 = w; Then, set y1, y2. If col < COL and meets the condition of y row_col <y row_col+1 , let y1 = y row_col+1 -y row_col , otherwise, let y1 = 1; If col < COL and meets the condition of [y row_col +|M(row,col) yLR |]<[y row_col+1 +|M(row,col+1) yLR |], let y2 = |M(row,col) yLR |-{[y row_col+1 +|M(row,col+1) yLR |]-[y row_col +|M(row,col) yLR |]}, otherwise, let y2 = h; Then, based on y1 and y2, we get the height difference of the overlapping area of ​​the two images in the Y direction, which is recorded as h. yDelta ; h yDelta =y2-y1+1 Finally, perform the following image fusion of the overlapping area of ​​the two images in the Y direction: <h2 style=";text-align:left;direction:ltr"> <h2 style=";text-align:left;direction:ltr"> x∈[x1,x2],y∈[y1,y2] After completion, if col < COL and it meets [y row_col +|M(row, col) yUD |] > [y row_col+1 +|M(row, col + 1) yUD |], then directly overwrite the corresponding area of the current image Img row_col onto the original large image Img Whole in the following manner: <h2 style=";text-align:left;direction:ltr"> <h2 style=";text-align:left;direction:ltr"> x∈[x1,x2],y∈[y1,y2] Step (4-5-4): Use QtConcurrent::run method to execute the image fusion task of the second type of overlapping parts of the two images in row COL images, where the second type of overlapping parts of the two images refers to the parts that will affect the fusion effect of the overlapping areas of the four images; Step (4-5-5): Get the starting coordinates of the image at row row and column col, recorded as (x start ,y start ); Step (4-5-6): If row>1 and col>1, all images in row are merged in the X-direction in the overlapping area of ​​the two images. Perform the following operations: First, set x1 and x2, let x1 = 1; let x2 = |M(row,col) xLR |; Then, set y1, y2, if [y row_col-1 +|M(row,col-1) yUD |]<[y row_col +|M(row,col) yUD |], let y1 = |M(row,col) yUD |-{[y row_col +|M(row,col) yUD |-[y row_col-1 +|M(row,col-1) yUD |]}, otherwise, let y1=|M(row,col) yUD |;y2=h; Then, based on x1 and x2, we get the height difference of the overlapping area of ​​the two images in the X direction, which is recorded as w xDelta : w xDelta =x2-x1+1 Finally, perform image fusion of the overlapping area of ​​the two images in the X direction: <h2 style=";text-align:left;direction:ltr"> <h2 style=";text-align:left;direction:ltr"> x∈[x1,x2],y∈[y1,y2] Step (4-5-7): Complete the image fusion of the overlapping area of ​​the two images in the row row COL images to obtain the second part of the image.

6. A method for stitching microscopic images based on improved linear fusion as claimed in claim 3, characterized in that The specific method for obtaining the third part of the image in step (4) is: Step (4-6-1): Use QtConcurrent::run method to perform the image fusion task of the four overlapping areas of the row-th row image that is not the first column; take the current image as the position reference object, record the upper right corner of the adjacent left image as the first overlapping area, record the upper left corner of the current image as the second overlapping area, and record the original large image Img Whole The local overlapping area of ​​the four images in the filled image is recorded as the third overlapping area; Step (4-6-2): Obtain a local image of the first overlapping area, denoted as Img1 row_col-1 ; First, set x1 and x2, let x1 = w-|M(row,col) xLR |+1; let x2=w; Then, set y1, y2, if y row_col-1 <y row_col , let y1 = y row_col -y row_col-1 +1, otherwise, let y1=1; if [y row_col-1 +|M(row,col-1) yUD |]>[y row_col +|M(row,col) yUD |], let y2=|M(row, col) yUD |-{[y row_col-1 +|M(row,col-1) yUD |]-[y row_col +|M(row,col) yUD |]}+1, otherwise, let y2=|M(row,col-1) yUD |+1; Finally, perform the following image copy operation to obtain the local image Img1 of the first overlapping area: row_col-1 : x∈[x1,x2],y∈[y1,y2]where Img1 row_col-1 (x k ,y k ) represents the local image Img1 of the first overlapping area row_col-1 The coordinate (x k ,y k ) pixel value, Img row_col-1 (x k ,y k ) represents the local image Img of the first overlapping area row_col-1 The pixel value at coordinate (x, y) in ; Step (4-6-3): Obtain a local image of the second overlapping area, denoted as Img2 row_col ; First, set x1 and x2, let x1 = 1; let x2 = |M(row,col) xLR |; Then, set y1, y2, if y row_col-1 >y row_col , let y1 = y row_col-1 -y row_col +1, otherwise, let y1=1; if [y row_col-1 +|M(row,col-1) yUD |]<[y row_col +|M(row,col) yUD |], let y2=|M(row, col) yUD |-{[y row_col +|M(row,col) yUD |]-[y row_col-1 +|M(row,col-1) yUD |]}+1, otherwise, let y2=|M(row,col) yUD |+1; Finally, perform the following image copy operation to obtain the local image Img2 of the second overlapping area: row_col : x∈[x1,x2],y∈[y1,y2]where Img2 row_col (x k ,y k ) represents the local image Img2 of the second overlapping area row_col The coordinate (x k ,y k ) pixel value, Img row_col (x k ,y k ) represents the local image Img of the second overlapping area row_col The pixel value at coordinate (x, y) in ; Step (4-6-4): The local image Img1 of the first overlapping area row_col-1 The local image Img2 of the second overlapping area row_col Perform linear image fusion along the X direction to obtain a temporary fused image, denoted as Img temp_row_col , the width is denoted as w temp_row_col , the height is denoted as h temp_row_col , the local image Img1 of the first overlapping area row_col-1 , the local image Img2 of the first overlapping area row_col And the temporary fusion image Img temp_row_col The sizes are all the same; x∈[x1,x2],y∈[y1,y2] Step (4-6-5): Temporary fusion image Img temp_row_col With the original large image Img Whole The corresponding overlapping areas are linearly merged along the Y direction: First, set x1 and x2, and let x1 = x row_col ; Let x2=x1+w temp_row_col -1; Then, set y1, y2, if x row_col-1 >x row_col , let x1=x row_col-1 , otherwise, let x1 = x row_col ; Let x2=x1+w temp_row_col -1; x∈[x1,x2],y∈[y1,y2] Step (4-6-6): Complete the image fusion of the overlapping areas of the four images in the rowth row, and obtain the third part of the image.

Images