A fast splicing method of unordered images based on improved binary tree model

By improving the binary tree model, using differential images and similarity rates to match disordered images, the registration index table is constructed, which solves the problem of low stitching efficiency of disordered images and achieves efficient image stitching and quality assurance.

CN114612304BActive Publication Date: 2024-11-22CHONGQING UNIV OF POSTS & TELECOMM
View PDF 7 Cites 0 Cited by

Patent Information

Application Number
CN202210242863.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-03-11
Publication Date
2024-11-22
Estimated Expiration
2042-03-11

AI Technical Summary

Technical Problem

The existing disordered image stitching technology is inefficient and requires frequent feature point extraction and matching, resulting in a large amount of meaningless work and it is difficult to quickly find image matching pairs.

Method used

Using an improved binary tree model, by normalizing preprocessing and random sorting of disordered images, differential images and similarity rates are obtained, images are matched by similarity rates, and registration index tables are constructed to reduce the number of searches for image matching pairs and improve stitching efficiency. .

Benefits of technology

It significantly improves the efficiency of disordered image stitching, reduces the time consumption of feature extraction and matching, ensures the quality of image stitching, and realizes efficient stitching of multiple disordered images.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114612304B_ABST
    Figure CN114612304B_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of image splicing, and in particular to a method for fast splicing of disordered images based on an improved binary tree model. The method comprises the following steps: normalizing and preprocessing a plurality of disordered images and then randomly sorting them to obtain an image sequence; obtaining a difference image, a similarity and a matching sequence based on the image sequence; matching the matching sequence with a plurality of reference images in order from large to small similarity, finding image matching pairs for splicing, and obtaining a plurality of panoramic sub-images; improving a binary tree, and preparing a registration index table based on the plurality of panoramic sub-images to assist the binary tree in splicing images to obtain a final panoramic image, constructing a matching sequence by calculating the similarity between each image and other image matching areas, reducing the number of invalid matches in searching for image matching, and recording the splicing information as a registration index table after a plurality of disordered images are paired and spliced, thereby eliminating the process of searching for image matching pairs in subsequent splicing.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to the technical field of image splicing, and in particular to a method for fast splicing of disordered images based on an improved binary tree model. Background Art

[0002] Image stitching has always been a hot topic in the field of computer vision. This technology is to stitch multiple low-resolution and narrow-angle images taken in the same scene into a panoramic image. The relatively mature sequential image stitching technology is widely used in remote sensing imaging, video surveillance, virtual reality, and 3D reconstruction. However, there are fewer studies on disordered image stitching technology, and there are still many unresolved issues in terms of accuracy and efficiency.

[0003] The X-ray digital imaging technology in the industrial field is limited by the size of flat-panel detectors. When collecting larger or longer objects, it is necessary to collect images of the workpiece at multiple angles. The images collected by different devices are bound to be in a messy state. The disordered image stitching technology can synthesize them into a large image containing the complete workpiece, which provides assistance for subsequent inspections. To stitch multiple disordered images, the key is how to accurately and quickly find image pairs with overlapping areas, and most of the current research directly finds matching images through feature point extraction and matching. After finding image pairs with overlapping areas and stitching them together to obtain panoramic sub-images, it is still necessary to find image matching pairs again to continue stitching, and a lot of meaningless work such as feature extraction and matching is repeated, which is inefficient. Summary of the invention

[0004] The purpose of the present invention is to provide a method for fast splicing of disordered images based on an improved binary tree model, aiming to reduce the time consumption of finding image matching pairs and greatly improve the efficiency of splicing of disordered images.

[0005] To achieve the above object, the present invention provides a method for fast splicing of disordered images based on an improved binary tree model, comprising the following steps:

[0006] After normalization preprocessing, several disordered images are randomly sorted to obtain an image sequence;

[0007] Acquire a difference image, a similarity rate and a matching sequence based on the image sequence;

[0008] Matching the matching sequence with a number of reference images in descending order of similarity, finding matching pairs of images and stitching them together to obtain a number of panoramic sub-images;

[0009] The binary tree is improved, and a registration index table is prepared based on a number of the panoramic sub-images to assist the binary tree in stitching images, so as to obtain a final panoramic image.

[0010] The specific method of obtaining the image sequence by normalizing and preprocessing a number of disordered images and then randomly sorting them is as follows:

[0011] Read several unordered images and select the image with the most identical width and height as the size standard;

[0012] Adjusting and normalizing the sizes of other images according to the size standard;

[0013] The normalized images are randomly sorted to obtain the image sequence.

[0014] The specific method of obtaining the difference image, similarity and matching sequence based on the image sequence is as follows:

[0015] Taking each image in the image sequence as a reference image and subtracting it from all the remaining images to obtain a number of difference images;

[0016] The plurality of difference images are quantified and the similarities of the difference areas are estimated respectively, the plurality of similarities are stored corresponding to the plurality of reference images, and the images are sorted according to the similarities to obtain a matching sequence.

[0017] The specific method of subtracting each image in the image sequence as a reference image from all remaining images to obtain a number of difference images is as follows:

[0018] Binarize the right side of the reference image and the left side of the remaining image to obtain the pixel value of the reference image and the pixel value of the image to be matched respectively;

[0019] Subtract the reference image pixel value from the to-be-matched image pixel value, and calculate the absolute value of the difference to obtain a difference image pixel value;

[0020] The pixel values ​​of the difference image are restored to obtain a plurality of the difference images.

[0021] Among them, the binary tree is improved, and a registration index table is prepared based on several panoramic sub-images to assist the binary tree in stitching images to obtain the final panoramic image. Specific method:

[0022] Record the stitching information of several panoramic sub-images to obtain a registration index table;

[0023] The binary tree is improved, and the registration index table is used to assist the binary tree in stitching images to obtain a final panoramic image.

[0024] The invention discloses a method for fast splicing of disordered images based on an improved binary tree model. The method pre-processes a plurality of disordered images by normalization and performs random sorting to obtain an image sequence. Based on the image sequence, a difference image, a similarity and a matching sequence are obtained. The matching sequence is matched with a plurality of reference images in order from large to small similarity to find image matching pairs for splicing to obtain a plurality of panoramic sub-images. The binary tree is improved and a registration index table is prepared based on the plurality of panoramic sub-images to assist the binary tree in splicing the images to obtain a final panoramic image. The partitioning strategy is used, that is, only half of the size of the original image is registered each time. On the basis of this method, that is, the matching area, this method calculates the similarity between each image and the matching area of ​​other images, and constructs a matching sequence according to the size of the similarity, thereby reducing the number of invalid matches in the process of finding image matching pairs. After multiple unordered images are paired and spliced, the splicing information is recorded as a registration index table, which eliminates the process of finding image matching pairs in subsequent splicing. The unordered image fast stitching algorithm of the improved binary tree model can greatly improve the efficiency of finding image matching pairs and the entire unordered image stitching process while ensuring the quality of image stitching, thereby achieving efficient stitching of multiple unordered images. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0026] Figure 1 It is a schematic diagram that two images do not have overlapping areas.

[0027] Figure 2 It is a schematic diagram that two images have overlapping areas.

[0028] Figure 3 It is a schematic diagram of the process of finding image matching pairs among 4 disordered images and the stitching results.

[0029] Figure 4 It is a schematic diagram of the process of finding image matching pairs among 4 disordered images and the stitching results.

[0030] Figure 5 There are 8 unordered images.

[0031] Figure 6 There are 4 unordered images.

[0032] Figure 7 This is a schematic diagram of the time required for the three methods to complete the stitching of disordered images.

[0033] Figure 8It is a schematic diagram of the disordered image splicing result completed by the disordered image fast splicing method of the improved binary tree model provided by the present invention.

[0034] Fig. 9 It is a schematic diagram of the improved binary tree splicing model.

[0035] Fig.10 The present invention provides a flowchart of a method for fast splicing of disordered images using an improved binary tree model.

[0036] Fig.11 It is a flowchart of a specific method of obtaining an image sequence by normalizing and preprocessing a number of disordered images and then randomly sorting them.

[0037] Fig.12 It is a flowchart of a specific method of obtaining a difference image, a similarity and a matching sequence based on the image sequence.

[0038] Fig.13 It is a flowchart of a specific method of subtracting each image in the image sequence as a reference image from all remaining images to obtain a number of difference images.

[0039] Fig.14 The invention is a flowchart of a specific method of improving a binary tree and preparing a registration index table based on several panoramic sub-images to assist the binary tree in stitching images to obtain a final panoramic image. DETAILED DESCRIPTION

[0040] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.

[0041] See also Figures 1 to 14 The present invention provides a method for fast splicing of disordered images based on an improved binary tree model, comprising the following steps:

[0042] S1 normalizes and preprocesses several disordered images and then randomly sorts them to obtain an image sequence;

[0043] Specifically, the unordered image set is traversed, the width and height of each image are read respectively, the width and height with the largest number are selected as the standard width and height, the sizes of other images are adjusted, the sizes of the images are normalized, and the images are numbered starting from 1 regardless of their contents, and are used as binary tree leaf nodes.

[0044] Specific method:

[0045] S11 reads a number of disordered images and selects the image with the most times of same width and height as the size standard;

[0046] S12 normalizes the sizes of other images according to the size standard;

[0047] S13 randomly sorts the normalized images to obtain the image sequence.

[0048] S2 obtains a difference image, a similarity and a matching sequence based on the image sequence;

[0049] Specific method:

[0050] S21: taking each image in the image sequence as a reference image and subtracting it from all the remaining images to obtain a plurality of difference images;

[0051] Specific method:

[0052] S211 binarizes the right side of the reference image and the left side of the residual image to obtain pixel values ​​of the reference image and pixel values ​​of the image to be matched respectively;

[0053] Specifically, generally, if two images have an overlapping area of ​​30% to 70%, they are considered to match each other. In order to balance accuracy and efficiency, the calculation area is limited to 50% (hereinafter referred to as the matching area).

[0054] S212: subtract the reference image pixel value from the to-be-matched image pixel value, and calculate the absolute value of the difference to obtain a difference image pixel value;

[0055] Specifically, because we need to stitch images horizontally, we start from the image numbered 1, take each image as the reference image, take its right matching area and the left matching area of ​​all the remaining images for binarization, and then perform a subtraction operation to obtain a difference image. The subtraction calculation is shown in the formula:

[0056] p(i,j)=|g r (i,j)-g t (i,j)|

[0057] Among them, p(i,j) is the pixel value of the difference image at (i,j), g r (i,j) and g t (i, j) represent the pixel values ​​of the reference image and the image to be matched at (i, j) respectively; the formula uses the method of subtracting and then finding the absolute value, because the subtraction of existing images will directly default to 0, so that no difference can be seen at all, and the entire difference image may even be completely black. Figure 1 In the above figure, it can be seen manually that the two images do not have overlapping areas, and the pixel distribution of the obtained difference image is relatively chaotic. Figure 2 The two images in have overlapping areas, so the pixel distribution of the difference image is simple.

[0058] S213 restores the pixel values ​​of the difference image to obtain a plurality of the difference images.

[0059] S22 quantifies the plurality of difference images and estimates the similarities of the difference areas respectively, stores the plurality of similarities in correspondence with the plurality of reference images, and sorts them by the similarities to obtain a matching sequence.

[0060] Specifically, after obtaining the difference image, it is necessary to quantify the difference. The present invention will quantify the difference by calculating the mean of the difference image, and then obtain the similarity of the matching area of ​​the two images subjected to the subtraction operation according to the formula. The calculation of the similarity is shown in the formula:

[0061]

[0062]

[0063] Among them, σ m,n It represents the mean value of the difference image after subtracting the matching area of ​​two images numbered m and n. M and N represent the length and width of the image respectively. m,n represents the similarity of the matching area between image m and image n. The 128 in the formula is an empirical value obtained after a large number of experiments. Put the original number and similarity of the reference image and the image to be subtracted together, and arrange the similarities in descending order. Each reference image and a series of similarities and corresponding image numbers form a matching sequence. Figure 3 Take the four disordered images in as an example, assuming that they are numbered 1 to 4 from left to right. Through the calculation of the above formula, the similarity between the matching areas of each image is shown in Table (1):

[0064]

[0065]

[0066] Table (1)

[0067] S3 matches the matching sequence with a number of reference images in order of similarity from large to small, finds image matching pairs and directly splices them to obtain a number of panoramic sub-images;

[0068] Specifically: The leftmost part of each row in the table is the matching area of ​​the reference image, and the remaining few in this row are the similarities between the matching area of ​​the image to be matched and the matching area of ​​the reference image, which have been sorted in descending order according to the similarity. Each row is taken as a matching sequence. The A-KAZE algorithm is used to extract feature points from the matching area of ​​the reference image and the first image in the matching sequence. The bidirectional KNN and RANSAC algorithms are used for feature matching. If the number of matching feature point pairs reaches 13 (13 is an empirical value after a large number of experiments), they are considered to have overlapping areas and the next matching sequence will be started directly; if it does not reach 13, the above operations are performed on the next image in the matching sequence in order; if the matching sequence is completed and there is no image that matches the reference image, then this reference image has no image with overlapping areas, and the next matching sequence will be continued. After all matching sequences are matched, all image pairs and spliced ​​panoramic sub-images are obtained. The above method is used in Figure 3 and Figure 4 Finding matching pairs of images only takes half the time of extracting and matching feature points from the entire image.

[0069] S4 improves the binary tree, and prepares a registration index table based on the plurality of panoramic sub-images to assist the binary tree in stitching images, so as to obtain a final panoramic image.

[0070] S41 records the stitching information of the plurality of panoramic sub-images to obtain a registration index table;

[0071] S42 improves the binary tree, and uses the registration index table to assist the binary tree in stitching images to obtain a final panoramic image.

[0072] Specifically, in the first stage of disordered splicing, image pairs with overlapping areas are found, and they are spliced ​​in pairs to obtain panoramic sub-images. In order to continue to complete the remaining image splicing, the panoramic sub-image needs to be used as an input image, and the image matching pair is found again through the extraction and matching of feature points. This operation is repeated with the first stage. The binary tree splicing model does not have a clear method to quickly find image pairs with overlapping areas in the panoramic sub-image set. The image numbering performed at the beginning is throughout. Therefore, after obtaining the panoramic sub-image, the original image numbers of the rightmost and leftmost sides of the panoramic sub-image and the original image numbers of the matching area participating in the splicing can be stored as an image registration index table to assist subsequent image splicing. The present invention will use all input images as leaf nodes to construct a binary tree from bottom to top. After completing the image splicing at each layer to obtain the panoramic sub-image, a registration index table is generated. The next layer only needs to directly infer the image pairs with overlapping areas based on the registration index table, without the need to extract and match feature points, and finally obtain a panoramic image at the root node.

[0073] by Figure 5As an example, the image registration index table is generated. The eight images are used as leaf nodes of the improved binary tree model. Assume that the leaf nodes are numbered from left to right as G1 to G8, G i L and G i R represents the original image G i The left matching area and the right matching area, that is, the matching areas on both sides, then the representation method of the panoramic sub-image is as follows:

[0074]

[0075] Among them, G ij is a panoramic sub-image, symbol Represents a splicing operation.

[0076] The registration index table generated by splicing 8 images in pairs is shown in Table 2:

[0077]

[0078] It can be concluded from the table that image G 47 G4R and G7L are involved in the splicing, and G 14 The far right is G4R, G 72 The leftmost side of G7L is G7L, so the image G 47 Only as a transition image, image G 14 and G 72 have overlapping areas. Similarly, G 38 As G 53 and G 86 The transition image, image G 53 and G 86 There are overlapping areas. The next layer of stitching only needs to find matching image pairs based on this registration index table, eliminating a lot of feature extraction and feature matching work for finding image matching pairs. Each subsequent layer of stitching can generate a registration index table, and the next layer can be stitched based on it. In other words, the entire unordered image stitching only needs to find an image matching pair once through feature matching.

[0079] In order to verify that the method proposed in the present invention can achieve efficient disordered image splicing, the present invention uses Figure 5 The 8 unordered images in Figure 6 The four disordered images in the image are spliced. The performance is compared with the sort-first-then-splicing method and the splicing method based on the binary tree model. The experimental comparison results are shown in Figure 7As shown, from the comparison results, it can be seen that the present invention has greatly improved the efficiency of splicing multiple disordered images. Compared with the two groups of images through the traditional method of sorting first and then splicing, the method of the present invention has greatly improved the efficiency. The time consumed by the method of the present invention for splicing 4 disordered images is 1 / 2 of the time consumed by the binary tree model, and the time consumed for splicing 8 disordered images is 1 / 3 of the time consumed by the binary tree model. As the number of images increases, the efficiency improvement of the method proposed by the present invention will be more obvious. In order to ensure that the present invention can still complete the splicing with high quality while improving the efficiency, Figure 8 The present invention demonstrates Figure 5 and Figure 6 The splicing result of the disordered images in Figure 8 It can be seen from the stitching results in that the method of the present invention can still ensure the quality of image stitching, and the panoramic image is smooth and continuous, with almost no distortion. Therefore, the method of quickly finding image matching pairs and the improvement of the binary tree stitching model by the registration index table proposed in the present invention can not only greatly improve the efficiency of disordered image stitching, but also ensure the quality of the image stitching result.

[0080] What is disclosed above is only a preferred embodiment of the present invention, and it certainly cannot be used to limit the scope of rights of the present invention. Ordinary technicians in this field can understand that all or part of the processes of the above embodiment and equivalent changes made according to the claims of the present invention still fall within the scope of the invention.

Claims

1. A method for fast splicing of disordered images based on an improved binary tree model, characterized in that: The following steps are involved: After normalization preprocessing, several disordered images are randomly sorted to obtain an image sequence; Acquire a difference image, a similarity rate and a matching sequence based on the image sequence; Matching the matching sequence with a number of reference images in descending order of similarity, finding matching pairs of images and stitching them together to obtain a number of panoramic sub-images; The binary tree is improved, and a registration index table is prepared based on a number of the panoramic sub-images to assist the binary tree in stitching images, so as to obtain a final panoramic image.

2. The method for fast splicing of disordered images based on an improved binary tree model as claimed in claim 1, characterized in that: The specific method of obtaining an image sequence by normalizing and preprocessing a number of disordered images and then randomly sorting them is as follows: Read several unordered images and select the image with the most identical width and height as the size standard; Adjusting and normalizing the sizes of other images according to the size standard; The normalized images are randomly sorted to obtain the image sequence.

3. The method for fast splicing of disordered images based on an improved binary tree model as claimed in claim 1, characterized in that: The specific method of obtaining the difference image, similarity and matching sequence based on the image sequence is: Taking each image in the image sequence as a reference image and subtracting it from all the remaining images to obtain a number of difference images; The plurality of difference images are quantified and the similarities of the difference areas are estimated respectively, the plurality of similarities are stored corresponding to the plurality of reference images, and the images are sorted according to the similarities to obtain a matching sequence.

4. The method for fast splicing of disordered images based on an improved binary tree model as claimed in claim 3, characterized in that: The specific method of subtracting each image in the image sequence as a reference image from all remaining images to obtain a plurality of difference images is as follows: Binarize the right side of the reference image and the left side of the remaining image to obtain the pixel value of the reference image and the pixel value of the image to be matched respectively; Subtract the reference image pixel value from the image pixel value to be matched, and calculate the absolute value of the difference to obtain a difference image pixel value; The pixel values ​​of the difference image are restored to obtain a plurality of the difference images.

5. The method for fast splicing of disordered images based on an improved binary tree model as claimed in claim 1, characterized in that: The improved binary tree and the registration index table prepared based on the plurality of panoramic sub-images to assist the binary tree in stitching images to obtain the final panoramic image are as follows: Record the stitching information of several panoramic sub-images to obtain a registration index table; The binary tree is improved, and the registration index table is used to assist the binary tree in stitching images to obtain a final panoramic image.

Citation Information

Patent Citations

  • Mutual information and interaction-based medical image splicing method

    CN103236048A

  • Document image binarization method based on support vector machine

    CN106096610A

  • Image registration and multi-resolution fusion-based panoramic image splicing method

    CN108416732A

  • Image splicing method based on binary tree

    CN109754366A

  • Measuring and calculating method, device and equipment for farmland area of aerial image, and medium

    CN111080526A