Image splicing method and device, equipment and medium

Abstract

The invention discloses an image splicing method and device, equipment and a medium, and relates to the technical field of image processing, and the method comprises the steps: obtaining a first image and a second image to be spliced, extracting feature points, obtaining a matching point set of the two images through feature point matching, then, the spatial internal consistency of the matching points is verified through clustering and cross screening, so that mismatching points which are accidentally matched due to feature similarity and are scattered in spatial distribution in a close-range large-parallax scene can be accurately eliminated, and effective matching point pairs with extremely high spatial consistency can be screened out even if overlapping regions are few; secondly, extracting an optimal matching point pair from the successfully verified point pairs through displacement consistency analysis, generating an optimal matching point subset, and further ensuring that the subset is not only high in accuracy, but also capable of accurately reflecting the overall spatial transformation relation of the two graphs; and finally, image splicing is performed based on the optimal matching point subset, so that the stability of the splicing process is improved, and the accuracy of the splicing result is improved.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to an image stitching method, apparatus, device, and medium. Background Technology

[0002] In the field of image stitching, close-up shooting, scenes with significant parallax, and limited overlapping areas between images pose severe challenges to stitching technology. Traditional stitching methods typically rely on feature point extraction and matching, along with outlier filtering. However, this approach exhibits significant shortcomings in the aforementioned scenarios: the feature point matching process is prone to leaving mismatched points, and even successfully matched feature point pairs often lack good consistency due to originating from objects at different depths within the scene; furthermore, the homography matrix calculated based on these local features is difficult to adapt to the stitching requirements of the entire image, ultimately resulting in poor stitching quality. Summary of the Invention

[0003] The purpose of this invention is to provide an image stitching method, apparatus, device, and medium that can solve the problems of inaccurate feature point matching and the inability of local homography matrix to replace whole image stitching when there are few overlapping areas with large parallax at close range.

[0004] To address the aforementioned technical problems, this invention provides an image stitching method, comprising: Obtain the first and second images to be stitched together, and extract feature points from the first and second images respectively; The feature points extracted from the first image and the second image are matched to obtain the matching point set of the first image and the matching point set of the second image; Clustering is performed on the matching point set of the first image and the matching point set of the second image respectively, and the spatial internal consistency of the matching points is verified by cross-filtering. The optimal matching point pair is extracted from the successfully verified matching point pair using the displacement consistency method, and an optimal matching point subset containing the corresponding feature points in the first image and the second image is generated. The first image and the second image are stitched together using the optimal matching point subset.

[0005] To address the aforementioned technical problems, the present invention also provides an image stitching device, comprising: The feature point extraction module is used to acquire the first image and the second image to be stitched together, and to extract feature points from the first image and the second image respectively. The feature point matching module is used to match the feature points extracted from the first image and the second image to obtain the matching point set of the first image and the matching point set of the second image. The clustering verification module is used to cluster the matching point set of the first image and the matching point set of the second image respectively, and verify the spatial internal consistency of the matching points through cross-filtering. The optimal point pair extraction module is used to extract the optimal matching point pairs from the verified matching point pairs using a displacement consistency method, and generate an optimal matching point subset containing the corresponding feature points in the first image and the second image. An image stitching module is used to stitch the first image and the second image together using the optimal matching point subset.

[0006] To address the aforementioned technical problems, the present invention also provides an electronic device, comprising: Memory, used to store computer programs; A processor is used to implement the steps of the image stitching method described above when executing the computer program.

[0007] To address the aforementioned technical problems, the present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the image stitching method described above.

[0008] As can be seen from the above technical solution, the image stitching method provided by the present invention includes: acquiring a first image and a second image to be stitched, and extracting feature points from the first image and the second image respectively; matching the feature points extracted from the first image and the second image to obtain a matching point set of the first image and a matching point set of the second image; clustering the matching point set of the first image and the matching point set of the second image respectively, and verifying the spatial internal consistency of the matching points through cross-filtering; extracting the optimal matching point pair from the successfully verified matching point pair using the displacement consistency method, generating an optimal matching point subset containing corresponding feature points in the first image and the second image; and stitching the first image and the second image using the optimal matching point subset.

[0009] The beneficial effects of this invention are as follows: The image stitching method provided by this invention, after acquiring the first and second images to be stitched and extracting feature points, first obtains the matching point set of the two images through feature point matching, and then verifies the spatial internal consistency of the matching points through clustering and cross-filtering. This can accurately eliminate mismatched points in close-range, large parallax scenes due to accidental matching of feature similarity and scattered spatial distribution. Even with few overlapping areas, it can screen out effective matching point pairs with extremely strong spatial consistency. Subsequently, the optimal matching point pair is extracted from the verified point pairs through displacement consistency analysis to generate the optimal matching point subset, further ensuring that the subset not only has a high accuracy rate but also accurately reflects the overall spatial transformation relationship of the two images. Finally, image stitching is performed based on the optimal matching point subset, which can effectively avoid stitching deviations caused by high initial matching noise and low effective information, making the stitching process more stable and the stitching result more accurate. This solves the problems of high difficulty and low reliability in image stitching in close-range, large parallax scenes with few overlapping areas.

[0010] In addition, the present invention also provides corresponding image stitching devices, electronic devices and computer-readable storage media for image stitching methods, which have the same or corresponding technical features as the image stitching methods mentioned above, and have the same effects. Attached Figure Description

[0011] To more clearly illustrate the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 A flowchart of the image stitching method provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of the image stitching device provided in an embodiment of the present invention. Detailed Implementation

[0013] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of the present invention.

[0014] It should be noted that, in the description of this invention, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. The terms "first," "second," etc., used in this invention are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0015] To enable those skilled in the art to better understand the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0016] The specific application environment architecture or specific hardware architecture on which the image stitching method depends is described here.

[0017] The embodiments of the present invention provide an image stitching method, and the method is described in detail in conjunction with the execution flow of the image stitching method. Figure 1 A flowchart of the image stitching method provided in the embodiments of the present invention is shown below. Figure 1 As shown, the method includes: S101. Obtain the first image and the second image to be stitched together, and extract feature points from the first image and the second image respectively.

[0018] In implementation, the first step is to acquire a first image and a second image to be stitched together. These images can be taken from multiple positions by a single camera, or from different positions by multiple cameras. Then, feature points are extracted from each image. These feature points represent key structural information about objects in the images. Methods for extracting feature points can include Scale-Invariant Feature Transform (SIFT), SpeededUp Robust Features (SURF), or Oriented Fast and Rotated BRIEF (ORB) algorithms. Alternatively, deep learning methods such as Superpoint and D2-Net networks can be used to extract feature points, creating 128-dimensional or 256-dimensional feature vectors for each feature point in both images.

[0019] S102. Match the feature points extracted from the first image and the second image to obtain the matching point set of the first image and the matching point set of the second image.

[0020] In implementation, after feature point extraction, feature point matching is used to find the most likely corresponding feature points in the second image for some feature points in the first image. These paired points are then categorized, forming the matching point sets for the first image and the second image. These two point sets not only build upon the previous feature extraction results but also establish a preliminary bridge for the spatial relationship between the two images, providing basic data for subsequent verification of the effectiveness of the matching points and calculation of relative image displacement. Feature point matching methods can include nearest neighbor search, brute-force matching, etc.

[0021] S103. Cluster the matching point set of the first image and the matching point set of the second image respectively, and verify the spatial internal consistency of the matching points through cross-filtering.

[0022] It should be noted that spatial internal consistency refers to matching points within the same cluster following similar geometric distribution patterns or sharing consistent spatial constraints in the two-dimensional space of their respective images. Cross-filtering is a strategy that uses bidirectional association verification to eliminate false matches and retain true matches from the initially matched point set. This invention clusters the matching point sets of the two images separately, filtering out effective point clusters (representing feature points of the same object or region) with high spatial clustering in each image; then, cross-filtering verifies the spatial internal consistency of the matching points, laying a reliable foundation for subsequently extracting the optimal matching point pairs.

[0023] S104. Extract the optimal matching point pair from the successfully verified matching point pairs using the displacement consistency method, and generate the optimal matching point subset containing the corresponding feature points in the first image and the second image.

[0024] It should be noted that displacement consistency refers to matching point pairs within the same target region following similar displacement vectors in two images. This is achieved by filtering out matching point pairs with highly consistent displacement vectors. This invention uses displacement consistency analysis (such as determining whether the displacement between point pairs conforms to the overall offset pattern of the two images) to eliminate abnormal displacement point pairs caused by local interference from spatially consistent matching point pairs, extracting the optimal matching point pairs with unified displacement patterns. The resulting optimal subset of matching points contains corresponding feature points from both images and possesses extremely high accuracy, providing a core basis for subsequent calculation of relative image displacement and achieving precise stitching.

[0025] S105. Use the optimal matching point subset to stitch the first image and the second image together.

[0026] It should be noted that the optimal matching point subset, due to its high accuracy and strong spatial consistency, can provide a reliable basis for calculating the horizontal and vertical displacement of the two images, thereby achieving accurate pixel alignment of the two images and ultimately outputting a complete and naturally transitioning stitched image.

[0027] In the image stitching method provided by the embodiments of the present invention, after acquiring the first image and the second image to be stitched and extracting feature points, the matching point set of the two images is first obtained through feature point matching. Then, the spatial internal consistency of the matching points is verified through clustering and cross-filtering. This can accurately eliminate mismatched points in close-range, large parallax scenes due to accidental matching of feature similarity and scattered spatial distribution. Even with few overlapping areas, effective matching point pairs with strong spatial consistency can be screened out. Subsequently, the optimal matching point pair is extracted from the verified point pairs through displacement consistency analysis to generate the optimal matching point subset. This further ensures that the subset not only has a high accuracy rate but also accurately reflects the overall spatial transformation relationship of the two images. Finally, image stitching is performed based on the optimal matching point subset, which can effectively avoid stitching deviations caused by high initial matching noise and low effective information, making the stitching process more stable and the stitching result more accurate. This solves the problems of high difficulty and low reliability in image stitching in close-range, large parallax scenes with few overlapping areas.

[0028] Furthermore, in a specific implementation, in the image stitching method provided in the embodiments of the present invention, step S102, which matches the feature points extracted from the first image and the second image to obtain a matching point set for the first image and a matching point set for the second image, may specifically include: matching the feature points extracted from the first image and the second image, calculating the ratio of the nearest neighbor distance to the second nearest neighbor distance for each feature point, and filtering out point pairs whose ratio of the nearest neighbor distance to the second nearest neighbor distance is less than a preset threshold; using the feature points belonging to the first image in the filtered point pairs as the matching point set for the first image; and using the feature points belonging to the second image in the filtered point pairs as the matching point set for the second image.

[0029] In implementation, by calculating the distance ratio between the nearest and second nearest neighbors of a feature point and setting a threshold for filtering, the essence is to eliminate unreliable matches based on the difference in feature similarity. When the nearest neighbor match of a feature point is much better than the second nearest neighbor match, it indicates that the match reliability is high. The resulting matching point sets in the two graphs retain potential corresponding points with highly similar features while initially filtering out matching points with too low similarity. This provides higher-quality basic data for subsequent spatial consistency verification, effectively reduces noise interference in subsequent processing, and improves the efficiency and accuracy of the overall matching process.

[0030] Furthermore, in specific implementation, in the above steps, the feature points extracted from the first image and the second image are matched, the ratio of the nearest neighbor distance to the second nearest neighbor distance of each feature point is calculated, and point pairs with a ratio of nearest neighbor distance to second nearest neighbor distance less than a preset threshold are selected. Specifically, this may include: performing a first matching operation on the feature points extracted from the first image and the second image, and determining the number of coarse matching point pairs based on the similarity of feature vectors; if the number of coarse matching point pairs is greater than a preset value (e.g., 4), then the first image and the second image are determined to be overlapping images, and a second matching operation is performed on the feature points extracted from the first image and the second image; calculating the ratio of the nearest neighbor distance to the second nearest neighbor distance of each feature point, selecting point pairs with a ratio of nearest neighbor distance to second nearest neighbor distance less than a preset threshold, and eliminating mismatched points with inconsistent geometric positions through geometric constraints; if the number of coarse matching point pairs is less than or equal to a preset value (e.g., 4), then the first image and the second image are determined to be non-overlapping images, and the stitching operation ends.

[0031] In implementation, the above process first uses feature vector similarity to quickly determine whether two images overlap. Then, for images determined to overlap, matching points are further refined through distance ratio filtering and geometric constraints. The geometric constraints can employ the Random Sample Consensus (RANSAC) algorithm. This approach both quickly eliminates non-overlapping images through coarse matching, avoiding invalid subsequent operations, and effectively removes geometrically inconsistent coarse matching points in overlapping image matching through distance ratio filtering and geometric constraints. This improves the efficiency of the stitching process while ensuring matching accuracy.

[0032] Furthermore, in a specific implementation, in the image stitching method provided in the embodiments of the present invention, step S103 clusters the matching point set of the first image and the matching point set of the second image respectively, and verifies the spatial internal consistency of the matching points through cross-filtering. Specifically, this may include: performing cluster analysis on the matching point set of the first image to filter out effective point clusters in the spatial distribution set of the first image; performing cluster analysis on the matching point set of the second image to filter out effective point clusters in the spatial distribution set of the second image; performing cross-filtering based on the pre-established initial matching relationship, retaining matching point pairs that simultaneously belong to the effective point clusters of the first image and the effective point clusters of the second image, and using them as verified successful matching point pairs. If no matching point pairs simultaneously belonging to the effective point clusters of the first image and the effective point clusters of the second image are found after cross-filtering, then the matching point pairs obtained by geometric constraint filtering in the initial matching relationship are used as verified successful matching point pairs.

[0033] It should be noted that the above clustering methods can employ density-based spatial clustering of applications with noise (DBSCAN), K-means clustering with mean shift, or hierarchical clustering, etc. This invention first filters valid point clusters with concentrated spatial distribution from the matching point sets of the two graphs through clustering (removing scattered outliers), and then combines the initial matching relationship to cross-filter point pairs that simultaneously belong to valid point clusters in both graphs. Essentially, this verifies the reliability of matching points from the perspective of spatial distribution rationality. This accurately removes mismatched points that are randomly matched due to feature similarity but have no spatial correlation, ensuring that successfully verified point pairs possess both feature similarity and conform to the spatial correlation logic of the two graphs. This provides high-quality data with extremely strong spatial consistency for subsequent extraction of optimal matching points and calculation of image displacement. In extreme cases, such as when the overlapping area is extremely small or the point clusters are scattered and there are no intersecting point pairs, this invention retains the matching point pairs that have been filtered by geometric constraints in the early stage as the verification result. This can avoid interruption of the stitching process, ensure the strictness of spatial consistency verification, effectively deal with the problem of scarce matching points in complex scenarios such as large parallax at close range and few overlapping areas, and improve the stability and fault tolerance of the entire image stitching process.

[0034] In implementation, DBSCAN clustering is performed on the matching point set of the first image; DBSCAN clustering is also performed on the matching point set of the second image using the same parameters; then, for the clustering results of the first and second images, the point sets corresponding to the largest cluster size or the top 60% of the cluster density are retained (ensuring that the retained points satisfy the spatial distribution rationality within a single image); subsequently, in the above-mentioned retained point sets, point pairs that still maintain the initial matching relationship are further screened out; finally, the retained matching point pairs must simultaneously meet two core conditions: first, the spatial distribution condition, that is, they belong to the main clusters after DBSCAN clustering in their respective images, and the spatial distribution conforms to the structure of the real scene; second, the initial matching condition, that is, they belong to the successfully matched point pairs verified by the previous geometric constraint filtering, and the geometric relationship is correct; for the matching point pairs that meet these two conditions, their point coordinates in the initial matching relationship after geometric constraint filtering are directly used as the input data for subsequent tasks.

[0035] The following uses shelf images as an example to illustrate the process of clustering cross-validation: First, a multi-category shelf detection model is trained using a deep learning network with multiple categories of data, such as shelf goods, shelves, and price tags. Then, based on the trained detection model, matching points in the first and second shelf images that are not within the detected product, shelf, and price tag areas are removed. Subsequently, among the remaining matching points, matching point pairs that are simultaneously located within the Region of Interest (ROI) of both the first and second shelf images are further retained. If no matching point set that meets the above criteria is found after the above filtering, the matching points in the initial matching relationship are used.

[0036] Furthermore, in a specific implementation, in the image stitching method provided in the embodiments of the present invention, step S104 uses displacement consistency to extract the optimal matching point pair from the successfully verified matching point pairs, generating an optimal matching point subset containing corresponding feature points in the first image and the second image. Specifically, this may include: calculating the lateral and longitudinal displacements between feature points in the first image and corresponding feature points in the second image based on the successfully verified matching point pairs, obtaining the displacement modulus of each matching point pair; statistically analyzing the displacement modulus data of all matching point pairs, generating a corresponding displacement modulus distribution histogram; fitting the displacement modulus distribution histogram using a Gaussian distribution model to obtain the probability distribution curve of the displacement modulus; selecting the matching point pair corresponding to the displacement modulus with the highest threshold from the fitted Gaussian distribution, and using it as the optimal matching point pair; summarizing all the selected optimal matching point pairs to form an optimal matching point subset containing corresponding feature points in the first image and the second image.

[0037] In implementation, firstly, the displacement and corresponding displacement modulus are calculated for each pair of matching points, using the following formula: ; ; ; ; in, For the index of the matching point, Match index for graph A x-coordinate Match index for graph B x-coordinate Match index for graph A The ordinate, Match index for graph B The ordinate, For index Calculate the displacement. For index The Euclidean distance of the matching points This represents the difference between the x-axis components. This represents the difference between the components of the vertical axis.

[0038] Then, all displacement moduli are statistically analyzed to generate corresponding displacement moduli distribution histograms. Next, a Gaussian distribution model is used to fit the displacement moduli distribution histograms to obtain the probability distribution curves of the displacement moduli; the formula for the Gaussian distribution is as follows: ; in, The center of displacement distribution Standard deviation, This refers to the amplitude.

[0039] Finally, the point with the largest threshold in the Gaussian distribution is selected as the best matching point pair for splicing.

[0040] This invention precisely extracts the optimal matching point pairs from successfully verified point pairs by quantifying displacement, statistically analyzing distribution, and combining Gaussian fitting, thus selecting the point pairs that best represent the overall transformation relationship. Through displacement modulus analysis and probabilistic model fitting, abnormal displacement point pairs caused by local interference can be effectively eliminated, ensuring that the final optimal matching point subset has both high accuracy and strong displacement consistency. This provides core and reliable data support for subsequent calculations of accurate relative offsets between two images and for achieving stable stitching, especially suitable for complex scenarios prone to local displacement deviations, such as close-range, large parallax.

[0041] Furthermore, in a specific implementation, in the image stitching method provided in the embodiments of the present invention, step S105 stitches the first image and the second image using the optimal matching point subset, which may specifically include: calculating the vertical displacement and horizontal displacement between the first image and the second image based on the optimal matching point subset; determining the relative offset relationship between the first image and the second image in spatial position according to the vertical displacement and horizontal displacement between the first image and the second image; and stitching the first image and the second image according to the relative offset relationship.

[0042] In implementation, this invention uses an optimal matching point subset as its core basis. First, it calculates the longitudinal and lateral displacements to determine the specific quantitative relationship of the spatial offset between the two images, and then performs stitching according to this relationship. This highly reliable optimal matching point approach accurately calculates the true relative offset between the two images, avoiding stitching misalignment caused by displacement errors. When stitching according to the determined offset relationship, it efficiently determines the canvas size and image alignment position. Combined with subsequent overlapping area processing, it ultimately achieves a natural transition and high-precision stitching effect. Especially in close-range, high-parallax situations, it can perform stitching without perspective transformation, optimizing stitching misalignment and excessive distortion caused by large parallax, effectively avoiding ghosting phenomena.

[0043] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method.

[0044] Embodiments of the present invention also provide an image stitching device. Figure 2 This is a schematic diagram of the image stitching device provided in an embodiment of the present invention. This embodiment is based on the perspective of functional modules, such as… Figure 2 As shown, the device includes: The feature point extraction module 10 is used to acquire the first image and the second image to be stitched together, and to extract feature points from the first image and the second image respectively. The feature point matching module 11 is used to match the feature points extracted from the first image and the second image to obtain the matching point set of the first image and the matching point set of the second image. Clustering verification module 12 is used to cluster the matching point set of the first image and the matching point set of the second image respectively, and verify the spatial internal consistency of the matching points through cross-filtering. The optimal point extraction module 13 is used to extract the optimal matching point pair from the verified matching point pair using the displacement consistency method, and generate the optimal matching point subset containing the corresponding feature points in the first image and the second image. Image stitching module 14 is used to stitch the first image and the second image using the optimal matching point subset.

[0045] In the image stitching device provided in the embodiments of the present invention, the interaction of the above four modules can accurately eliminate mismatched points in close-range, high-parallax scenes due to accidental matching of features and scattered spatial distribution. Even with few overlapping areas, it can screen out effective matching point pairs with strong spatial consistency. This ensures that the subset not only has a high accuracy rate but also accurately reflects the overall spatial transformation relationship of the two images. It effectively avoids stitching deviations caused by high initial matching noise and low effective information, making the stitching process more stable and the stitching result more accurate. This solves the problems of high difficulty and low reliability in image stitching in close-range, high-parallax scenes with few overlapping areas.

[0046] Since the embodiments of the image stitching device and the image stitching method correspond to each other, the descriptions of the features in the embodiments corresponding to the image stitching device can be found in the relevant descriptions of the embodiments corresponding to the image stitching method, and will not be repeated here. Furthermore, it has the same beneficial effects as the image stitching method mentioned above.

[0047] Furthermore, in a specific implementation, in the image stitching device provided in the embodiments of the present invention, the feature point matching module 11 can be specifically used to match the feature points extracted from the first image and the second image, calculate the ratio of the nearest neighbor distance to the second nearest neighbor distance of each feature point, and filter out point pairs whose ratio of the nearest neighbor distance to the second nearest neighbor distance is less than a preset threshold; the feature points belonging to the first image in the filtered point pairs are used as the matching point set of the first image; and the feature points belonging to the second image in the filtered point pairs are used as the matching point set of the second image.

[0048] Furthermore, in a specific implementation, in the image stitching device provided in the embodiments of the present invention, the clustering verification module 12 can be specifically used to perform clustering analysis on the matching point set of the first image to filter out effective point clusters in the spatial distribution set of the first image; perform clustering analysis on the matching point set of the second image to filter out effective point clusters in the spatial distribution set of the second image; perform cross-filtering based on the pre-established initial matching relationship, retain matching point pairs that belong to both the effective point clusters of the first image and the effective point clusters of the second image, and use them as successfully verified matching point pairs; if no matching point pairs that belong to both the effective point clusters of the first image and the effective point clusters of the second image are selected after cross-filtering, then the matching point pairs obtained by geometric constraint filtering in the initial matching relationship are used as successfully verified matching point pairs.

[0049] Furthermore, in a specific implementation, in the image stitching device provided in the embodiments of the present invention, the optimal point pair extraction module 13 can be specifically used to calculate the lateral and longitudinal displacements between feature points in the first image and corresponding feature points in the second image based on the successfully verified matching point pairs, to obtain the displacement modulus of each matching point pair; to statistically analyze the displacement modulus data of all matching point pairs and generate a corresponding displacement modulus distribution histogram; to fit the displacement modulus distribution histogram using a Gaussian distribution model to obtain the probability distribution curve of the displacement modulus; to select the matching point pair corresponding to the displacement modulus with the highest threshold in the fitted Gaussian distribution and use it as the optimal matching point pair; and to summarize all the selected optimal matching point pairs to form an optimal matching point subset containing the corresponding feature points in the first and second images.

[0050] Furthermore, in a specific implementation, in the image stitching device provided in the embodiments of the present invention, the image stitching module 14 can be specifically used to calculate the longitudinal displacement and lateral displacement between the first image and the second image based on the optimal matching point subset pair; determine the relative offset relationship between the first image and the second image in spatial position according to the longitudinal displacement and lateral displacement between the first image and the second image; and stitch the first image and the second image according to the relative offset relationship.

[0051] Embodiments of the present invention also provide an electronic device, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the steps in any of the above-described image stitching method embodiments.

[0052] Embodiments of the present invention also provide a computer-readable storage medium storing a computer program configured to execute the steps in any of the above-described image stitching method embodiments at runtime.

[0053] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0054] Embodiments of the present invention also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in any of the above-described image stitching method embodiments.

[0055] Embodiments of the present invention also provide another computer program product, including a non-volatile computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps in any of the above-described image stitching method embodiments.

[0056] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0057] The image stitching method, apparatus, device, and medium provided by the present invention have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only intended to help understand the method and core ideas of the present invention. It should be noted that those skilled in the art can make various improvements and modifications to the present invention without departing from its principles, and these improvements and modifications also fall within the protection scope of the present invention.

Claims

1. An image stitching method, characterized in that, include: Obtain the first and second images to be stitched together, and extract feature points from the first and second images respectively; The feature points extracted from the first image and the second image are matched to obtain the matching point set of the first image and the matching point set of the second image; Clustering is performed on the matching point set of the first image and the matching point set of the second image respectively, and the spatial internal consistency of the matching points is verified by cross-filtering. The optimal matching point pair is extracted from the successfully verified matching point pair using the displacement consistency method, and an optimal matching point subset containing the corresponding feature points in the first image and the second image is generated. The first image and the second image are stitched together using the optimal matching point subset.

2. The image stitching method according to claim 1, characterized in that, Clustering is performed on the matching point set of the first image and the matching point set of the second image respectively, and the spatial internal consistency of the matching points is verified by cross-filtering, including: Cluster analysis is performed on the matching point set of the first image to filter out the effective point clusters in the spatial distribution set of the first image; Cluster analysis is performed on the matching point set of the second image to filter out the effective point clusters in the spatial distribution set of the second image; Cross-filtering is performed based on the pre-established initial matching relationship, retaining matching point pairs that belong to both the valid point clusters of the first image and the valid point clusters of the second image, and these are considered as successfully verified matching point pairs.

3. The image stitching method according to claim 2, characterized in that, After cross-filtering based on pre-established initial matching relationships, the process also includes: If no matching point pair belonging to both the first image and the second image is found after cross-filtering, the matching point pair obtained by geometric constraint filtering in the initial matching relationship is used as the verified matching point pair.

4. The image stitching method according to claim 1, characterized in that, The optimal matching point pair is extracted from the successfully verified matching point pairs using a displacement consistency method, generating an optimal matching point subset containing corresponding feature points from the first image and the second image, including: Based on the successfully verified matching point pairs, calculate the lateral and longitudinal displacements between the feature points in the first image and the corresponding feature points in the second image to obtain the displacement magnitude of each matching point pair; Collect the displacement modulus data of all matching point pairs and generate the corresponding displacement modulus distribution histogram; The displacement modulus distribution histogram is fitted using a Gaussian distribution model to obtain the probability distribution curve of the displacement modulus. In the fitted Gaussian distribution, the matching point pair corresponding to the displacement modulus with the highest threshold is selected as the optimal matching point pair; All the selected optimal matching point pairs are aggregated to form an optimal matching point subset containing the corresponding feature points in the first image and the second image.

5. The image stitching method according to claim 1, characterized in that, The first image and the second image are stitched together using the optimal matching point subset, including: Based on the optimal matching point subset, calculate the longitudinal and lateral displacements between the first image and the second image; The relative offset relationship between the first image and the second image in spatial position is determined based on the longitudinal and lateral displacements between the first image and the second image. The first image and the second image are stitched together according to the relative offset relationship.

6. The image stitching method according to claim 1, characterized in that, The feature points extracted from the first image and the second image are matched to obtain the matching point set of the first image and the matching point set of the second image, including: The feature points extracted from the first image and the second image are matched, the ratio of the nearest neighbor distance to the second nearest neighbor distance of each feature point is calculated, and point pairs with a ratio of the nearest neighbor distance to the second nearest neighbor distance less than a preset threshold are selected. The feature points belonging to the first image in the selected point pairs are used as the matching point set of the first image; The feature points belonging to the second image in the selected point pairs are used as the matching point set of the second image.

7. The image stitching method according to claim 6, characterized in that, The feature points extracted from the first image and the second image are matched, and the ratio of the nearest neighbor distance to the second nearest neighbor distance for each feature point is calculated. Point pairs whose ratio of nearest neighbor distance to second nearest neighbor distance is less than a preset threshold are selected, including: The feature points extracted from the first image and the second image are subjected to a first matching operation, and the number of coarse matching point pairs is determined based on the similarity of the feature vectors. If the number of coarse matching point pairs is greater than a preset value, the first image and the second image are determined to be overlapping images, and a second matching operation is performed on the feature points extracted from the first image and the second image; the ratio of the nearest neighbor distance to the second nearest neighbor distance of each feature point is calculated, point pairs with a ratio of the nearest neighbor distance to the second nearest neighbor distance less than a preset threshold are selected, and mismatched points with inconsistent geometric positions are eliminated through geometric constraints; If the number of coarse matching point pairs is less than or equal to the preset value, then the first image and the second image are determined to be non-overlapping images, and the stitching operation ends.

8. An image stitching device, characterized in that, include: The feature point extraction module is used to acquire the first image and the second image to be stitched together, and to extract feature points from the first image and the second image respectively. The feature point matching module is used to match the feature points extracted from the first image and the second image to obtain the matching point set of the first image and the matching point set of the second image. The clustering verification module is used to cluster the matching point set of the first image and the matching point set of the second image respectively, and verify the spatial internal consistency of the matching points through cross-filtering. The optimal point pair extraction module is used to extract the optimal matching point pairs from the verified matching point pairs using a displacement consistency method, and generate an optimal matching point subset containing the corresponding feature points in the first image and the second image. An image stitching module is used to stitch the first image and the second image together using the optimal matching point subset.

9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, configured to implement the steps of the image stitching method as described in any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the image stitching method as described in any one of claims 1 to 7.

Images