A panoramic image stitching and restoration system and method based on feature matching
By optimizing the panoramic image stitching order through feature matching and triple planning mechanisms, the problem of poor stitching quality in existing technologies is solved, and automated stitching and high-quality panoramic image restoration are achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- NANJING LIPADE DIGITAL TECH CO LTD
- Filing Date
- 2025-09-29
- Publication Date
- 2026-07-17
AI Technical Summary
In existing technologies, the fixed image stitching order leads to poor stitching quality, which cannot effectively improve the panoramic image restoration effect, and there is a lack of automated stitching order planning methods.
A panoramic image stitching and restoration system based on feature matching is adopted. Through the acquisition and processing of shooting information, image feature matching, image stitching order planning and panoramic image restoration module, the system optimizes the image stitching order by using a triple planning mechanism. It combines the number of feature points, the uniformity of distribution and image quality information to automatically plan the stitching order.
It effectively improves image stitching quality, reduces the accumulation of stitching errors, enhances the restoration effect of panoramic images, and realizes automated stitching sequence planning for panoramic images.
Smart Images
Figure CN121304498B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of image processing technology, specifically a panoramic image stitching and restoration system and method based on feature matching. Background Technology
[0002] Panoramic image stitching and restoration is a process that uses algorithms to synthesize several overlapping images into a seamless panoramic image and repair imperfections. The image stitching method often affects the subsequent restoration effect. Typically, images are stitched together one by one in the order they were captured. However, the quality of image stitching is easily affected by the number of matching feature points between adjacent images. A higher number of matching feature points means richer information about the overlapping area between the two images. Calculating the homography matrix using more matching points allows for a more accurate estimation of the geometric transformation relationship between the images, thus improving alignment accuracy. Therefore, increasing the number of matching feature points is crucial. Prioritizing the stitching of more adjacent images can effectively improve image stitching quality. However, existing technologies typically use the method of stitching images one by one according to the shooting order, which cannot effectively improve image stitching quality due to the fixed image stitching order. Secondly, if the image stitching order is planned based solely on the number of feature points, there may be more than one pair of adjacent images with the most matching feature points. In the face of this situation, there is a lack of a method to automatically plan the image stitching order to achieve automated image stitching. The existing image stitching methods cannot effectively improve stitching quality, which in turn affects the subsequent panoramic image restoration effect. Summary of the Invention
[0003] The purpose of this invention is to provide a panoramic image stitching and restoration system and method based on feature matching to solve the problems raised in the prior art.
[0004] To achieve the above objectives, the present invention provides the following technical solution: a panoramic image stitching and restoration system based on feature matching, the system comprising a shooting information acquisition and processing module, an image feature matching module, an image stitching sequence planning module, and a panoramic image restoration module; The image capture information acquisition and processing module is used to capture images and collect image capture information, and to number the images based on the collected image capture information. The image feature matching module is used to match feature points of two adjacent images and obtain image feature information; The image stitching order planning module is used to map the image number and image feature information to a rectangular coordinate system in stages, set a triple planning mechanism based on the staged mapping information, plan the image stitching order based on the triple planning mechanism, and perform image stitching processing according to the planned image stitching order. The panoramic image restoration module is used to restore the stitched image by using Photoshop tools to repair the stitched image and remove stitching marks.
[0005] Preferably, the shooting information acquisition and processing module includes an image shooting unit, a shooting order statistics unit, and an image numbering processing unit; The image capturing unit is used to capture images using a camera; The shooting order statistics unit is used to count the shooting order of images that need to be stitched and repaired. Images with adjacent shooting orders need to be stitched together. The image numbering processing unit is used to number the images according to the order in which they were captured: the first captured image is numbered 1, the second captured image is numbered 2, and the remaining images are numbered sequentially according to the order in which they were captured.
[0006] Preferably, the image feature matching module includes a feature descriptor extraction unit, a feature matching unit, and a feature information statistics unit; The feature descriptor extraction unit is used to detect feature points in the image using the SIFT algorithm and generate a feature vector descriptor for each feature point. The feature matching unit is used to calculate the Euclidean distance between descriptors of two adjacent images, set a distance threshold, and if the Euclidean distance between two descriptors is less than or equal to the distance threshold, it is determined that the feature points corresponding to the two descriptors match each other. The feature information statistics unit is used to count the number of matching feature points between all adjacent numbered images, the quality information of all images, and the distribution location information of feature points in all images.
[0007] Preferably, the image stitching order planning module includes an image information mapping unit, a triple planning mechanism setting unit, and an image stitching unit; The image information mapping unit is used to map the information of adjacent images to a Cartesian coordinate system by using the number of feature points that match each other in adjacent images as the first stitching order selection reference factor. The first circular region is divided in the coordinate system with the number of the two adjacent images as the center and the first stitching order selection reference factor as the radius. The horizontal coordinate of the center is the number of the first image taken in the two adjacent images, and the vertical coordinate of the center is the number of the second image taken in the two adjacent images. The triple planning mechanism setting unit is used to set up a triple planning mechanism for image stitching sequence planning. The image stitching unit is used to perform image stitching processing according to the planned image stitching sequence.
[0008] Preferably, the triple planning mechanism is set as follows: compare the areas of several first circular regions, select the circular region with the largest area; if there is only one selected first circular region, then set a first stitching planning mechanism: take the two images corresponding to the selected first circular region as the two images to be stitched together for the first time; if the number of selected first circular regions is greater than 1, analyze the uniformity of feature point distribution in the images corresponding to several selected first circular regions, and take the average uniformity of feature point distribution in any two adjacent images as the second stitching order selection reference factor for the second information mapping: use the adjacent image numbers Using the center and the second stitching order selection reference factor as the radius, a second circular region is divided in the coordinate system. The areas of the second circular regions are compared, and the region with the smallest area is selected. If only one second circular region is selected, a second stitching planning mechanism is set: the two images corresponding to the selected second circular region are used as the two images to be stitched in the first step. If the number of selected second circular regions is greater than one, the quality of the images corresponding to several selected second circular regions is checked, and the average quality of any two adjacent images is used as the third stitching order selection reference factor for the third information mapping: using the numbers of adjacent images as the center, ... The third stitching order selection reference factor is used to divide the coordinate system into third circular regions with a radius. The areas of these third circular regions are compared, and the region with the largest area is selected. A third stitching planning mechanism is set up: if only one third circular region is selected, the two images corresponding to this region are used as the first two images to be stitched; if more than one third circular region is selected, any two images corresponding to any selected region are selected as the first two images to be stitched. After selecting the first two images to be stitched, the triple planning mechanism is used cyclically to select the next two images to be stitched. Until the image stitching order is planned, the selected image information will no longer participate in the planning process of the triple planning mechanism. That is: if the two images selected for the first stitching are numbered 1 and 2, the first circular region with (1,2) as the center will be eliminated from several first circular regions. The two images to be stitched in the second time will be selected by comparing the area of the remaining first circular regions. If more than one circular region in the remaining first circular regions has the maximum area, the triple planning mechanism will continue to be used until the two images to be stitched in the second time are selected. This process continues until the stitching order of all images is selected. By numbering the captured images and counting the number of matching feature points between each pair of adjacent images, the numbering information and the number of feature points are visually mapped onto a Cartesian coordinate system. A first circular region is drawn in the coordinate system with the number of adjacent images as the center and the number of feature points as the radius. The area of this circular region is then compared visually; a larger area indicates a greater number of matching feature points between the corresponding two images. Based on this comparison, the two images to be stitched together are selected initially. Furthermore, considering that more than one circular region may have the largest area (meaning there may be more than one pair of adjacent images with the highest number of matching feature points), a second factor is incorporated into the stitching reference: the uniformity of feature point distribution in the image. Uniformly distributed feature points can more comprehensively cover the overlapping areas of the images. This approach avoids errors in homography matrix calculation caused by local features, thus improving image stitching quality. Therefore, after incorporating the second factor, information mapping is re-performed. The method of comparing circular area sizes is used to select the first two images for stitching. Considering that there may be more than one circular area with the largest size in the second circular area, a third reference factor—image quality information—is incorporated. The selection is performed using the same method. Under this triple-planning mechanism, the optimal image stitching order is planned, effectively reducing the accumulation of image stitching errors caused by stitching images sequentially according to the shooting order in existing technologies. This achieves automated stitching order planning for panoramic images, effectively improving stitching quality and reducing the impact of accumulated image stitching errors on subsequent image restoration, thereby enhancing the restoration effect of subsequent panoramic images.
[0009] A panoramic image stitching and restoration method based on feature matching includes the following steps: S100: Captures images and collects image capture information, and processes the images by numbering them based on the collected image capture information; S101: Perform feature point matching on two adjacent images and obtain image feature information; S102: Map the image number and image feature information to the Cartesian coordinate system in stages, set up a triple planning mechanism based on the staged mapping information, plan the image stitching order based on the triple planning mechanism, and perform image stitching processing according to the planned image stitching order. S103: Perform restoration processing on the stitched image.
[0010] Preferably, step S100 includes: taking images using a camera, counting that a total of n images have been taken, stitching and repairing the n images, and numbering the images according to the shooting order of the n images from first to last: setting the numbers of the n images as {1,2,...,n}.
[0011] Preferably, step S101 includes: detecting feature points in n images using the SIFT algorithm, generating a feature vector descriptor for each feature point, calculating the Euclidean distance between descriptors of two adjacent images, setting a distance threshold of D, and determining that the feature points corresponding to the two descriptors match if the Euclidean distance between the two descriptors is less than or equal to D, and counting the number of matching feature points in all adjacent images as F = {F1, F2, ..., F...} n-1}, where F1 represents the number of feature points that match between image number 2 and image number 1, the quality information of n images is detected, the quality refers to the signal-to-noise ratio of the image, and the distribution location information of feature points in the n images is collected.
[0012] Preferably, S102 includes: setting {F1,F2,...,F...} n-1 As the first stitching order selection reference factor, the numbering information of adjacent images is mapped to a Cartesian coordinate system: A first circular region is divided in the coordinate system with the numbers of the two adjacent images as the center and the first stitching order selection reference factor as the radius. A random first circular region is then defined with (i, i+1) as the center and F... i Given a circular region with radius F, the area of a randomly selected first circular region is π*(F). i ) 2 , i represents the number of the i-th image taken, F i This represents the number of feature points that match between the image numbered i+1 and the image numbered i. A total of n-1 first circular regions are defined. The areas of these n-1 first circular regions are compared, and the largest circular region is selected. If only one first circular region is selected, a first stitching mechanism is implemented: the two images corresponding to the selected first circular region are used as the first two images to be stitched. For example, if only one first circular region is selected, and the center of the first circular region is (1,2), then images numbered 1 and 2 are used as the first two images to be stitched, i.e., priority is given to stitching. The two images are stitched together. If the number of selected first circular regions is greater than 1, a total of k selected first circular regions are counted. The center of a random circular region among the k first circular regions is (j, j+1). The distribution information of feature points in images numbered j and j+1 is retrieved. A rectangular coordinate system is established with the center of each image as the origin. The coordinates of the feature points in the image are obtained. A convex hull that can cover all feature points is delineated. The Graham scan method is used to arrange the vertices of the convex hull in clockwise order. The area of images numbered j and j+1 are obtained as S. j and S j+1 Calculate the convex hull area (Area) that covers all feature points in the image numbered j using the following formula. j : ; Where m represents the total number of vertices that can cover the convex hull of all feature points in the image numbered j, e represents the vertex index, and x m+1 =x1, y m+1 =y1,x e and y e Let W represent the x and y coordinates of the e-th convex hull vertex arranged in sequence, respectively, to obtain the uniformity of the feature point distribution in the image numbered j. j , The convex hull area Area that covers all feature points in the image numbered j+1 is calculated. j+1 The uniformity of the feature point distribution in the image numbered j+1 is obtained as W. j+1 , The uniformity of the average distribution of feature points in images numbered j and j+1 is obtained as W, where W = (W j +W j+1 ) / 2, using W as a random second stitching order selection reference factor, when incorporating the second reference factor and analyzing the uniformity of feature point distribution in the image, the convex hull area ratio analysis method is used to analyze the uniformity of feature point distribution. This method can intuitively quantify the spatial coverage of feature points. Compared with analysis of variance, it is more suitable for handling large-scale feature point analysis, and the analysis method is more robust to the existence of local outliers, such as outliers. A second information mapping is performed: a random second circular region is divided in the coordinate system with (j,j+1) as the center and W as the radius. A total of k second circular regions are divided. The areas of the k second circular regions are compared, and the second circular region with the smallest area is selected. If there is only one selected second circular region, a second stitching planning mechanism is set: the two images corresponding to the selected second circular region are used as the two images to be stitched in the first time. If the number of selected second circular regions is greater than 1, a total of p selected second circular regions are counted. The center of a random circular region among the p second circular regions is (u,u+1). The detected image quality of the images numbered u and u+1 is SNR respectively. u SNR u+1 The average quality of the images numbered u and u+1 is SNR=(SNR u +SNR u+1) / 2, using SNR as a random third stitching order selection reference factor, perform the third information mapping: with (u,u+1) as the center and SNR as the radius, divide a random third circular region in the coordinate system, dividing a total of p third circular regions. Compare the areas of the p third circular regions, and select the third circular region with the largest area. Set the third stitching planning mechanism: if there is only one selected third circular region, then the two images corresponding to the selected third circular region are used as the two images to be stitched for the first time; if the number of selected third circular regions is greater than 1, select the two images corresponding to any selected third circular region as the two images to be stitched for the first time. After selecting the two images to be stitched for the first time, the triple planning mechanism is used cyclically to select the two images to be stitched for each subsequent time until the image stitching order planning is completed.
[0013] Compared with the prior art, the beneficial effects of the present invention are: This invention assigns numbers to captured images and counts the number of matching feature points between adjacent images. The numbering and feature point count information are then visually mapped onto a Cartesian coordinate system. A first circular region is defined in the coordinate system, centered on the image number and with the feature point count as the radius. The area of this circular region is then compared visually; a larger area indicates a greater number of matching feature points between the two images. Based on this comparison, the first two images to be stitched are selected. Furthermore, considering that more than one circular region may have the largest area (meaning more than one pair of adjacent images has the highest number of matching feature points), a second factor is incorporated into the stitching reference: the uniformity of feature point distribution in the image. Uniformly distributed feature points can more comprehensively cover the overlapping areas of the images. This approach avoids errors in homography matrix calculation caused by local features, thus improving image stitching quality. Therefore, after incorporating the second factor, information mapping is re-performed. The method of comparing the area of circular regions is used to select the two images for the first stitching. Considering that there may be more than one circular region with the largest area in the second circular region, a third reference factor, image quality information, is incorporated and selected in the same way. Under the triple planning mechanism, the optimal image stitching order is planned, which can effectively reduce the accumulation of image stitching errors caused by stitching images one by one in the shooting order of existing technologies. This achieves automated stitching order planning for panoramic images, effectively improving stitching quality and reducing the impact of accumulated image stitching errors on subsequent image restoration, thus helping to improve the restoration effect of subsequent panoramic images. When incorporating a second reference factor and analyzing the uniformity of feature point distribution in an image, the convex hull area ratio analysis method is used to analyze the uniformity of feature point distribution. This method can intuitively quantify the spatial coverage of feature points. Compared with analysis of variance, it is more suitable for handling large-scale feature point analysis, and the analysis method is more robust to the presence of local outliers, such as outliers. Attached Figure Description
[0014] Figure 1 This is a schematic diagram of the structure of a panoramic image stitching and restoration system based on feature matching according to the present invention; Figure 2 This is a flowchart illustrating a panoramic image stitching and restoration method based on feature matching according to the present invention. Detailed Implementation
[0015] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention. Example
[0016] like Figure 1 As shown, this embodiment provides a panoramic image stitching and restoration system based on feature matching. The system includes: a shooting information acquisition and processing module, an image feature matching module, an image stitching sequence planning module, and a panoramic image restoration module. The image capture information acquisition and processing module is used to capture images and collect image capture information, and to number the images based on the collected image capture information. The image feature matching module is used to match feature points between two images with adjacent numbers and obtain image feature information; The image stitching order planning module is used to map image numbers and image feature information to a Cartesian coordinate system in stages. Based on the staged mapping information, a triple planning mechanism is set up, and the image stitching order is planned according to the triple planning mechanism. The image stitching process is then performed according to the planned image stitching order. The panoramic image restoration module is used to restore stitched images by using Photoshop tools to repair stitched images and remove stitching marks.
[0017] The image capture and processing module includes an image capture unit, a capture sequence statistics unit, and an image numbering processing unit. The image capture unit is used to capture images using a camera. The capture sequence statistics unit is used to count the capture order of images that need to be stitched and repaired, and images with adjacent capture orders need to be stitched together. The image numbering processing unit is used to number the images according to their capture order: the first captured image is assigned the number 1, the second captured image is assigned the number 2, and the remaining images are numbered sequentially according to the order in which they were captured.
[0018] The image feature matching module includes a feature descriptor extraction unit, a feature matching unit, and a feature information statistics unit. The feature descriptor extraction unit is used to detect feature points in the image using the SIFT algorithm and generate a feature vector descriptor for each feature point. The feature matching unit is used to calculate the Euclidean distance between the descriptors of two adjacent images and set a distance threshold. If the Euclidean distance between two descriptors is less than or equal to the distance threshold, the feature points corresponding to the two descriptors are determined to be matched. The feature information statistics unit is used to count the number of matched feature points in all adjacent images, the quality information of all images, and the distribution location information of feature points in all images.
[0019] The image stitching order planning module includes an image information mapping unit, a triple planning mechanism setting unit, and an image stitching unit. The image information mapping unit uses the number of matching feature points between adjacent images as the first stitching order selection reference factor to map the information of adjacent images onto a Cartesian coordinate system. A first circular region is divided in the coordinate system with the numbers of the two adjacent images as the center and the first stitching order selection reference factor as the radius. The horizontal coordinate of the center is the number of the first image taken among the two adjacent images, and the vertical coordinate is the number of the second image taken among the two adjacent images. The triple planning mechanism setting unit is used to set up a triple planning mechanism for image stitching order planning. The image stitching unit is used to perform image stitching processing according to the planned image stitching order.
[0020] The triple planning mechanism is set up as follows: First, compare the areas of several first circular regions and select the largest circular region. If only one first circular region is selected, then the first stitching planning mechanism is set: the two images corresponding to the selected first circular region are used as the two images to be stitched together for the first time. If the number of selected first circular regions is greater than one, analyze the uniformity of feature point distribution in the images corresponding to several selected first circular regions, and use the average uniformity of feature point distribution in any two adjacent images as the second stitching priority selection reference factor for the second information mapping: Divide the coordinate system into second circular regions with the adjacent image numbers as the center and the second stitching priority selection reference factor as the radius. Compare the areas of the second circular regions and select the smallest second circular region. If only one second circular region is selected, then the second stitching planning mechanism is set: the two images corresponding to the selected second circular region are used as the two images to be stitched together for the first time. If the number of selected second circular regions is greater than one, check the quality of the images corresponding to several selected second circular regions, and use the average quality of any two adjacent images as the third stitching priority selection reference factor for the third information mapping: the third stitching planning mechanism is set up with the adjacent image numbers as the center and the second stitching priority selection reference factor as the radius for the second information mapping. Using the reference factor as the radius, a third circular region is divided in the coordinate system. The areas of these third circular regions are compared, and the region with the largest area is selected. A third stitching planning mechanism is then established: if only one third circular region is selected, the two images corresponding to that region are used as the first two images to be stitched; if more than one third circular region is selected, any two images corresponding to any selected region are selected as the first two images to be stitched. After selecting the first two images, this triple planning mechanism is used iteratively to select the next two images to be stitched until... Once the image stitching order is planned, the selected image information will no longer participate in the triple planning mechanism. That is, if the two images selected for the first stitching are numbered 1 and 2, the first circular region centered at (1,2) will be eliminated from the several first circular regions. The two images to be stitched in the second time will be selected by comparing the areas of the remaining first circular regions. If more than one circular region in the remaining first circular regions has the maximum area, the triple planning mechanism will continue to be used until the two images to be stitched in the second time are selected. This process continues until the stitching order of all images is selected. Example
[0021] like Figure 2 As shown, this embodiment provides a panoramic image stitching and restoration method based on feature matching, which is implemented based on the image stitching and restoration system in this embodiment, and specifically includes the following steps: S100: Capture images and collect image capture information, and number the images according to the collected image capture information: Capture images using the camera, count the current number of n images captured, and stitch and repair the n images. Number the images according to the shooting order of the n images from first to last: Set the numbers of the n images as {1,2,...,n}; S101: Perform feature point matching and obtain image feature information for two adjacent images: Use the SIFT algorithm to detect feature points in n images, generate a feature vector descriptor for each feature point, calculate the Euclidean distance between the descriptors of two adjacent images, set a distance threshold of D, and if the Euclidean distance between two descriptors is less than or equal to D, then the feature points corresponding to the two descriptors are considered to be matched. The set of the number of matched feature points in all adjacent images is F={F1,F2,...,F...} n-1}, where F1 represents the number of feature points that match between image number 2 and image number 1, the quality information of n images is detected, the quality refers to the signal-to-noise ratio of the image, and the distribution location information of feature points in the n images is collected; S102: Map image numbers and image feature information to a Cartesian coordinate system in stages. Based on this staged mapping information, set up a triple planning mechanism. Plan the image stitching order according to this mechanism, and perform image stitching processing according to the planned order: {F1, F2, ..., F...} n-1 As the first stitching order selection reference factor, the numbering information of adjacent images is mapped to a Cartesian coordinate system: A first circular region is divided in the coordinate system with the numbers of the two adjacent images as the center and the first stitching order selection reference factor as the radius. A random first circular region is then defined with (i, i+1) as the center and F... i Given a circular region with radius F, the area of a randomly selected first circular region is π*(F). i ) 2 , i represents the number of the i-th image taken, F iThis represents the number of feature points that match between the image numbered i+1 and the image numbered i. A total of n-1 first circular regions are defined. The areas of these n-1 first circular regions are compared, and the largest circular region is selected. If only one first circular region is selected, a first stitching mechanism is implemented: the two images corresponding to the selected first circular region are used as the first two images to be stitched. For example, if only one first circular region is selected, and the center of the first circular region is (1,2), then images numbered 1 and 2 are used as the first two images to be stitched, i.e., priority is given to stitching. The two images are stitched together. If the number of selected first circular regions is greater than 1, a total of k selected first circular regions are counted. The center of a random circular region among the k first circular regions is (j, j+1). The distribution information of feature points in images numbered j and j+1 is retrieved. A rectangular coordinate system is established with the center of each image as the origin. The coordinates of the feature points in the image are obtained. A convex hull that can cover all feature points is delineated. The Graham scan method is used to arrange the vertices of the convex hull in clockwise order. The area of images numbered j and j+1 are obtained as S. j and S j+1 Calculate the convex hull area (Area) that covers all feature points in the image numbered j using the following formula. j : ; Where m represents the total number of vertices that can cover the convex hull of all feature points in the image numbered j, e represents the vertex index, and x m+1 =x1, y m+1 =y1,x e and y e Let W represent the x and y coordinates of the e-th convex hull vertex arranged in sequence, respectively, to obtain the uniformity of the feature point distribution in the image numbered j. j , The convex hull area Area that covers all feature points in the image numbered j+1 is calculated. j+1 The uniformity of the feature point distribution in the image numbered j+1 is obtained as W. j+1 , The uniformity of the average distribution of feature points in images numbered j and j+1 is obtained as W, where W = (W j +W j+1) / 2, using W as a random second stitching order selection reference factor, a second information mapping is performed: a random second circular region is divided in the coordinate system with (j,j+1) as the center and W as the radius, resulting in k second circular regions. The areas of the k second circular regions are compared, and the second circular region with the smallest area is selected. If only one second circular region is selected, a second stitching planning mechanism is set: the two images corresponding to the selected second circular region are used as the two images to be stitched in the first step; if the number of selected second circular regions is greater than 1, a total of p selected second circular regions are counted, and the center of a random circular region among the p second circular regions is (u,u+1). The detected image quality of images numbered u and u+1 is SNR respectively. u SNR u+1 The average quality of the images numbered u and u+1 is SNR=(SNR u +SNR u+1 ) / 2, using SNR as a random third stitching order selection reference factor, perform the third information mapping: divide a random third circular region in the coordinate system with (u,u+1) as the center and SNR as the radius, and divide a total of p third circular regions. Compare the areas of the p third circular regions and select the third circular region with the largest area. Set the third stitching planning mechanism: if there is only one selected third circular region, then the two images corresponding to the selected third circular region are used as the two images to be stitched for the first time; if the number of selected third circular regions is greater than 1, select the two images corresponding to any selected third circular region as the two images to be stitched for the first time. After selecting the two images to be stitched for the first time, the triple planning mechanism is used cyclically to select the two images to be stitched for each subsequent time until the stitching order of the images is planned. For example, if there are 6 images to be stitched together, and the final image stitching order selected according to the triple planning mechanism is: images numbered 1 and 2 are stitched together, images numbered 3 and 4 are stitched together, images numbered 5 and 6 are stitched together, images numbered 2 and 3 are stitched together, and images numbered 4 and 5 are stitched together, then images numbered 1 and 2 are stitched together first, then images numbered 3 and 4 are stitched together, then images numbered 5 and 6 are stitched together, then the stitched images numbered 1 and 2 are stitched together with the stitched images numbered 3 and 4, and finally the stitched images numbered 1, 2, 3, and 4 are stitched together with the stitched images numbered 5 and 6 to obtain the final stitched panoramic image.
[0022] S103: Perform restoration processing on the stitched image.
[0023] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.
Claims
1. A panoramic image stitching and restoration system based on feature matching, characterized in that: The system includes: a shooting information acquisition and processing module, an image feature matching module, an image stitching sequence planning module, and a panoramic image restoration module; The image capture information acquisition and processing module is used to capture images and collect image capture information, and to number the images based on the collected image capture information. The image feature matching module is used to match feature points of two adjacent images and obtain image feature information; The image stitching order planning module is used to map the image number and image feature information to a rectangular coordinate system in stages, set a triple planning mechanism based on the staged mapping information, plan the image stitching order based on the triple planning mechanism, and perform image stitching processing according to the planned image stitching order. The panoramic image restoration module is used to restore the stitched image. The image stitching sequence planning module includes an image information mapping unit, a triple planning mechanism setting unit, and an image stitching unit; The image information mapping unit is used to map the information of adjacent images to a Cartesian coordinate system by using the number of feature points that match each other in adjacent images as the first stitching order selection reference factor. The first circular region is divided in the coordinate system with the number of the two adjacent images as the center and the first stitching order selection reference factor as the radius. The horizontal coordinate of the center is the number of the first image taken in the two adjacent images, and the vertical coordinate of the center is the number of the second image taken in the two adjacent images. The triple planning mechanism setting unit is used to set up a triple planning mechanism for image stitching sequence planning. The image stitching unit is used to perform image stitching processing according to the planned image stitching sequence; The triple planning mechanism is set up as follows: The areas of several first circular regions are compared, and the largest circular region is selected. If only one first circular region is selected, a first stitching planning mechanism is set: the two images corresponding to the selected first circular region are used as the two images to be stitched together for the first time. If the number of selected first circular regions is greater than one, the uniformity of feature point distribution in the images corresponding to several selected first circular regions is analyzed. The average uniformity of feature point distribution in any two adjacent images is used as a second stitching priority selection reference factor for the second information mapping: Second circular regions are divided in the coordinate system with the adjacent image numbers as the center and the second stitching priority selection reference factor as the radius. The areas of the second circular regions are compared, and the smallest second circular region is selected. If only one second circular region is selected, a second stitching planning mechanism is set: the two images corresponding to the selected second circular region are used as the two images to be stitched together for the first time. Two images; if the number of selected second circular regions is greater than 1, the quality of the images corresponding to several selected second circular regions is checked, and the average quality of any two adjacent images is used as the third stitching order selection reference factor for the third information mapping: the third circular region is divided in the coordinate system with the adjacent image numbers as the center and the third stitching order selection reference factor as the radius. The areas of the third circular regions are compared, and the third circular region with the largest area is selected. The third stitching planning mechanism is set: if there is only one selected third circular region, the two images corresponding to the selected third circular region are used as the first two images to be stitched; if the number of selected third circular regions is greater than 1, the two images corresponding to any selected third circular region are selected as the first two images to be stitched. After selecting the first two images to be stitched, the triple planning mechanism is used cyclically to select the next two images to be stitched until the image stitching order is planned.
2. The panoramic image stitching and restoration system based on feature matching according to claim 1, characterized in that: The shooting information acquisition and processing module includes an image shooting unit, a shooting sequence statistics unit, and an image numbering processing unit. The image capturing unit is used to capture images using a camera; The shooting order statistics unit is used to count the shooting order of the images that need to be stitched and repaired; The image numbering processing unit is used to number the images according to the order in which they were captured: the first captured image is numbered 1, the second captured image is numbered 2, and the remaining images are numbered sequentially according to the order in which they were captured.
3. The panoramic image stitching and restoration system based on feature matching according to claim 2, characterized in that: The image feature matching module includes a feature descriptor extraction unit, a feature matching unit, and a feature information statistics unit; The feature descriptor extraction unit is used to detect feature points in the image using the SIFT algorithm and generate a feature vector descriptor for each feature point. The feature matching unit is used to calculate the Euclidean distance between descriptors of two adjacent images, set a distance threshold, and if the Euclidean distance between two descriptors is less than or equal to the distance threshold, it is determined that the feature points corresponding to the two descriptors match each other. The feature information statistics unit is used to count the number of matching feature points between all adjacent numbered images, the quality information of all images, and the distribution location information of feature points in all images.
4. A panoramic image stitching and restoration method based on feature matching, characterized in that: Includes the following steps: S100: Captures images and collects image capture information, and processes the images by numbering them based on the collected image capture information; S101: Perform feature point matching on two adjacent images and obtain image feature information; S102: Map the image number and image feature information to the Cartesian coordinate system in stages, set up a triple planning mechanism based on the staged mapping information, plan the image stitching order based on the triple planning mechanism, and perform image stitching processing according to the planned image stitching order. S103: Perform restoration processing on the stitched image; S102 includes: setting {F1,F2,...,F...} n-1 As the first stitching order selection reference factor, F1 represents the number of feature points that match between image number 2 and image number 1, and n represents the number of images. The numbering information of adjacent images is mapped to a Cartesian coordinate system: a first circular region is divided in the coordinate system with the numbers of the two adjacent images as the center and the first stitching order selection reference factor as the radius. A random first circular region is defined with (i, i+1) as the center and F1 as the radius. i Given a circular region with radius F, the area of a randomly selected first circular region is π*(F). i ) 2 , i represents the number of the i-th image taken, F i Let represent the number of feature points that match the image numbered i+1 with the image numbered i. A total of n-1 first circular regions are defined. The areas of these n-1 first circular regions are compared, and the largest circular region is selected. If only one first circular region is selected, a first stitching planning mechanism is implemented: the two images corresponding to the selected first circular region are used as the two images to be stitched together for the first time. If the number of selected first circular regions is greater than 1, a total of k first circular regions are counted. The center of one random circular region among these k first circular regions is (j, j+1). The distribution information of feature points in the images numbered j and j+1 is retrieved. A Cartesian coordinate system is established with the center of each image as the origin. The coordinates of the feature points in the images are obtained. A convex hull that covers all feature points is defined. The Graham scan method is used to arrange the vertices of the convex hull in a clockwise order. The areas of the images numbered j and j+1 are obtained as S. j and S j+1 Calculate the convex hull area (Area) that covers all feature points in the image numbered j using the following formula. j : ； Where m represents the total number of vertices that can cover the convex hull of all feature points in the image numbered j, e represents the vertex index, and x m+1 =x1, y m+1 =y1,x e and y e Let W represent the x and y coordinates of the e-th convex hull vertex arranged in sequence, respectively, to obtain the uniformity of the feature point distribution in the image numbered j. j , The convex hull area Area that covers all feature points in the image numbered j+1 is calculated. j+1 The uniformity of the feature point distribution in the image numbered j+1 is obtained as W. j+1 , The uniformity of the average distribution of feature points in images numbered j and j+1 is obtained as W, where W = (W j +W j+1 ) / 2, using W as a random second stitching order selection reference factor, perform the second information mapping: divide a random second circular region in the coordinate system with (j,j+1) as the center and W as the radius, divide a total of k second circular regions, compare the areas of the k second circular regions, and select the second circular region with the smallest area. If there is only one selected second circular region, set the second stitching planning mechanism: use the two images corresponding to the selected second circular region as the two images to be stitched in the first time; If the number of selected second circular regions is greater than 1, and there are a total of p selected second circular regions, then the center of a random circular region among these p second circular regions is (u, u+1). The image qualities detected for regions numbered u and u+1 are SNR and SNR, respectively. u SNR u+1 The average quality of the images numbered u and u+1 is SNR=(SNR u +SNR u+1 ) / 2, using SNR as a random third stitching order selection reference factor, perform the third information mapping: with (u,u+1) as the center and SNR as the radius, divide a random third circular region in the coordinate system, dividing a total of p third circular regions. Compare the areas of the p third circular regions, and select the third circular region with the largest area. Set the third stitching planning mechanism: if there is only one selected third circular region, then the two images corresponding to the selected third circular region are used as the two images to be stitched for the first time; if the number of selected third circular regions is greater than 1, select the two images corresponding to any selected third circular region as the two images to be stitched for the first time. After selecting the two images to be stitched for the first time, the triple planning mechanism is used cyclically to select the two images to be stitched for each subsequent time until the image stitching order planning is completed.
5. The panoramic image stitching and restoration method based on feature matching according to claim 4, characterized in that: S100 includes: taking pictures using a camera, counting that a total of n pictures have been taken, stitching and repairing the n pictures, and numbering the pictures according to the shooting order of the n pictures from first to last: setting the numbers of the n pictures as {1,2,...,n}.
6. The panoramic image stitching and restoration method based on feature matching according to claim 5, characterized in that: S101 includes: detecting feature points in n images using the SIFT algorithm, generating a feature vector descriptor for each feature point, calculating the Euclidean distance between the descriptors of two adjacent images, setting a distance threshold of D, and determining that the feature points corresponding to the two descriptors match if the Euclidean distance between the two descriptors is less than or equal to D, and counting the number of matching feature points in all adjacent images as F={F1,F2,...,F...} n-1 }, where F1 represents the number of feature points that match between image number 2 and image number 1, the quality information of n images is detected, and the distribution location information of feature points in n images is collected.