An image stitching method, device, terminal equipment and storage medium
By identifying feature points on the same plane and calculating disparity differences during image stitching, aggregating feature points with small disparities, and processing planes with large disparities separately, the ghosting and stitching line problems in high-disparity image stitching are solved, improving stitching efficiency and quality.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- UBTECH ROBOTICS CORP LTD
- Filing Date
- 2022-06-15
- Publication Date
- 2026-06-02
Smart Images
Figure CN115205111B_ABST
Abstract
Description
Technical Field
[0001] This application belongs to the field of image processing technology, and in particular relates to an image stitching method, apparatus, terminal device and storage medium. Background Technology
[0002] Image stitching is a crucial research topic in computer vision. Its function is to stitch together two or more captured images into an ultra-wide-angle image. With technological advancements, image stitching methods have gradually matured, but existing methods suffer from the following problems:
[0003] 1. Current image stitching methods can solve the problem of stitching images with small parallax, but for stitching images with large parallax, the stitching results often show ghosting or obvious stitching line marks.
[0004] 2. The greater the disparity between the images to be stitched, the higher the complexity of the stitching algorithm required. However, existing stitching methods do not have the ability to predict the disparity size in advance. This leads to using image stitching methods that address large disparities to stitch images with small disparities, reducing stitching efficiency. Alternatively, it leads to using image stitching methods that address small disparities to stitch images with large disparities, resulting in poor stitching quality. Summary of the Invention
[0005] This application provides an image stitching method, apparatus, terminal device, and storage medium, which can solve the problems of low stitching efficiency and poor stitching effect in the prior art.
[0006] The first aspect of this application provides an image stitching method, including:
[0007] Obtain two images of the target scene to be stitched together, and the image feature points of each image to be stitched together;
[0008] Identify the image feature points belonging to the same plane in each of the images to be stitched together;
[0009] Based on the disparity values of each pair of mutually matching planes in the two images to be stitched, calculate the disparity difference between every two disparity values;
[0010] The image feature points of two planes in the same image to be stitched together with a disparity difference less than a difference threshold are aggregated into one class to obtain the target class cluster of each image to be stitched together;
[0011] Based on the target cluster of each of the images to be stitched, the two images to be stitched are stitched together to obtain the target image.
[0012] A second aspect of this application provides an image stitching apparatus, comprising: an acquisition unit for acquiring two images of a target scene to be stitched, and image feature points of each image to be stitched; a determination unit for determining the image feature points belonging to the same plane in each image to be stitched; a calculation unit for calculating a disparity difference between every two disparity values based on the disparity values of each set of mutually matching planes in the two images to be stitched; a clustering unit for aggregating the image feature points of two planes in the same image to be stitched whose disparity difference is less than a difference threshold into a class, thereby obtaining a target cluster for each image to be stitched; and a stitching unit for stitching the two images to be stitched together according to the target cluster of each image to be stitched, thereby obtaining a target image.
[0013] A third aspect of this application provides a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the above-described image stitching method.
[0014] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the above-described image stitching method.
[0015] The fifth aspect of this application provides a computer program product that, when run on a terminal device, causes the terminal device to execute the image stitching method described in the first aspect.
[0016] In the embodiments of this application, two images of the target scene to be stitched and image feature points of each image to be stitched are acquired; image feature points belonging to the same plane in each image to be stitched are determined; based on the disparity values of each pair of mutually matching planes in the two images to be stitched, the disparity difference between each pair of disparity values is calculated; the image feature points of two planes in the same image to be stitched whose disparity difference is less than the difference threshold are aggregated into one class to obtain the target class cluster of each image to be stitched; based on the target class cluster of each image to be stitched, the two images to be stitched are stitched together to obtain the target image. This allows the terminal device to adaptively perform image stitching based on the disparity size of the planes in the scene, separately considering each plane with a large disparity difference, realizing multi-plane stitching, avoiding ghosting or obvious stitching marks in the stitching result. At the same time, by merging the image feature points of planes with small disparity differences, it is not necessary to perform stitching processing on each plane with a small disparity difference separately, thus improving the efficiency of image stitching. Attached Figure Description
[0017] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0018] Figure 1 This is a schematic diagram illustrating the implementation process of an image stitching method provided in an embodiment of this application;
[0019] Figure 2 This is a schematic diagram of the image stitching process provided in an embodiment of this application;
[0020] Figure 3 This is a schematic diagram illustrating the specific implementation process of step S102 provided in the embodiments of this application;
[0021] Figure 4 This is a schematic diagram of the clustering process provided in an embodiment of this application;
[0022] Figure 5 This is a schematic diagram illustrating the specific implementation process of step S103 provided in the embodiments of this application;
[0023] Figure 6 This is a schematic diagram illustrating the specific implementation process of step S105 provided in the embodiments of this application;
[0024] Figure 7 This is a schematic diagram illustrating the specific implementation process of filling or deleting pixels provided in an embodiment of this application;
[0025] Figure 8 This is a schematic diagram illustrating the determination of the target adjustment area provided in an embodiment of this application;
[0026] Figure 9 This is a first schematic diagram illustrating the image stitching of images to be stitched using different algorithms, provided in an embodiment of this application.
[0027] Figure 10 This is a second schematic diagram illustrating the image stitching of images to be stitched using different algorithms, provided in an embodiment of this application.
[0028] Figure 11 This is a third schematic diagram provided in this application embodiment of image stitching using different algorithms on images to be stitched;
[0029] Figure 12 This is a bar chart showing testers voting on the stitching results produced by different image stitching methods, provided in the embodiments of this application.
[0030] Figure 13This is a schematic diagram of the structure of an image stitching device provided in an embodiment of this application;
[0031] Figure 14 This is a schematic diagram of the structure of the terminal device provided in the embodiments of this application. Detailed Implementation
[0032] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are protected by this application.
[0033] Existing image stitching methods have the following problems:
[0034] 1. Current image stitching methods can solve the problem of stitching images with small parallax, but for stitching images with large parallax, the stitching results often show ghosting or obvious stitching line marks.
[0035] 2. The greater the disparity between the images to be stitched, the higher the complexity of the stitching algorithm required. However, existing stitching methods do not have the ability to predict the disparity size in advance. This leads to using image stitching methods that address large disparities to stitch images with small disparities, reducing stitching efficiency. Alternatively, it leads to using image stitching methods that address small disparities to stitch images with large disparities, resulting in ghosting in the stitching result.
[0036] Based on this, this application proposes an image stitching method that can be used to solve the above problems.
[0037] To illustrate the technical solution of this application, specific embodiments are described below.
[0038] Figure 1 The illustration shows a schematic diagram of the implementation process of an image stitching method provided in an embodiment of this application. This method can be applied to terminal devices and is suitable for situations where it is necessary to improve image stitching efficiency and enhance image stitching effect.
[0039] The aforementioned terminal devices can be smart devices such as mobile phones, computers, and roadside units, and this application does not impose any restrictions on this.
[0040] Specifically, the above image stitching method may include the following steps S101 to S105.
[0041] Step S101: Obtain two images of the target scene to be stitched together, and the image feature points of each image to be stitched together.
[0042] In the embodiments of this application, the user can take pictures of the target scene to obtain several images to be stitched together. The image content of the images to be stitched together includes various objects in the target scene. The image content of different images to be stitched together has some similarities. Image stitching is based on the similarities between different images to be stitched together to combine the images into a panoramic image.
[0043] The target scene can be determined based on the user's selection. The image to be stitched is the image that needs to be stitched together to form a panoramic image.
[0044] It should be understood that when the number of images to be stitched is greater than two, the terminal device can first stitch two images to obtain the first panoramic image, then stitch the first panoramic image with the next image to be stitched to obtain the second panoramic image, and so on, until all images to be stitched are completed to obtain the final panoramic image. The image stitching between any two images can adopt the scheme provided in this application.
[0045] After acquiring two images of the target scene to be stitched together, the terminal device can perform feature point recognition on each image to obtain image feature points.
[0046] The feature point recognition can be implemented using Fast Feature Detection (FAST), Scale-invariant feature transform (SIFT), Speeded Up Robust Features (SURF), or other feature point detection algorithms, and this application does not impose any restrictions on this.
[0047] Step S102: Determine the image feature points belonging to the same plane in each image to be stitched.
[0048] In the embodiments of this application, the target scene includes one or more objects, and the surfaces of these objects can form different planes, such as walls, floors, etc. In order to improve the stitching effect, the terminal device needs to classify the extracted image feature points, determine the plane to which each image feature point belongs, obtain the image feature points belonging to the same plane in each image to be stitched, and then stitch the images according to the image feature points in the plane, respectively, according to the feature point matching pairs in each plane.
[0049] Specifically, after acquiring the image feature points of two images I0 and I1 to be stitched together, the terminal device can match the image feature points in the two images based on image information such as pixel values, coordinate positions, and the relative positional relationships between the image feature points, to obtain a series of feature point matching pairs. in, and, It is a pair of feature point matching pairs.
[0050] Accordingly, based on the feature point matching pairs and the plane to which each image feature point belongs, the terminal device can also perform matching for all planes within the two images I0 and I1 to be stitched together. A set of mutually matched planes is the same plane in the two images I0 and I1 to be stitched together.
[0051] Step S103: Calculate the disparity difference between every two disparity values based on the disparity values of each set of mutually matching planes in the two images to be stitched.
[0052] In the embodiments of this application, since the same plane is not in the same position in two images to be stitched, that is, each set of mutually matching planes has a corresponding disparity, the terminal device can calculate the disparity value of each set of mutually matching planes in the two images to be stitched based on the position of each plane in the corresponding image to be stitched, and then calculate the disparity difference between each two disparity values.
[0053] Among them, the disparity difference can represent the difference in disparity between two different planes in two images to be stitched together.
[0054] For example, if there are planes A, B, and C in the target scene, the terminal device can determine the planes A0, B0, and C0 in the image to be stitched (I0) that correspond to planes A, B, and C, respectively, and the planes A1, B1, and C1 in the image to be stitched (I1) that match planes A0, B0, and C0, respectively. Then, based on the position of plane A0 in the image to be stitched (I0) and the position of plane A1 in the image to be stitched (I1), the terminal device can calculate the disparity value D of the matching planes A0 and A1. A In the same way, the disparity values D of the matching planes B0 and B1 can be obtained. B The disparity value D of the matching planes C0 and C1 C .
[0055] After obtaining the disparity value, the terminal device can calculate the difference in disparity between plane A and plane B in the two images to be stitched, |D|. A -D B |D, the difference in disparity between plane B and plane C in the two images to be stitched together. B -D C| and, the difference in disparity between plane A and plane C in the two images to be stitched, |D A -D C |
[0056] Step S104: Aggregate the image feature points of two planes in the same image to be stitched together into one class to obtain the target class cluster of each image to be stitched together.
[0057] The target cluster is a set of image feature points after clustering in the image to be stitched, including image feature points of at least one plane in the image to be stitched.
[0058] In the embodiments of this application, if the disparity difference between two planes is less than the difference threshold, it means that the image stitching effect obtained by stitching the images based on the two planes is similar. The terminal device can aggregate the image feature points of the two planes into one class to obtain the target cluster corresponding to the two planes.
[0059] If the disparity difference between two planes is greater than or equal to the difference threshold, it means that the image stitching effect obtained by stitching the images based on these two planes is quite different. The terminal device needs to process them separately. In this case, the image feature points of the two planes can be aggregated into one class according to the plane to which they belong, so as to obtain the target clusters corresponding to the two planes respectively.
[0060] The difference threshold can be set according to the actual situation.
[0061] After clustering the image feature points in the two images I0 and I1 to be stitched according to the disparity difference, the target clusters of image I0 and I1 can be obtained. It should be understood that since the disparity value is a common attribute of the two images I0 and I1, after clustering, the target clusters of image I0 and I1 correspond one-to-one.
[0062] Step S105: Based on the target cluster of each image to be stitched, stitch the two images to be stitched together to obtain the target image.
[0063] In the embodiments of this application, the terminal device can use an image stitching model to perform image stitching once based on each pair of corresponding target clusters in the two images to be stitched, to obtain the stitching results corresponding to each pair of corresponding target clusters, and to determine the target image from the stitching results.
[0064] The target image is a panoramic image obtained by stitching together two images to be stitched together.
[0065] Specifically, assuming the target cluster obtained by clustering the image feature points of the image I0 to be stitched is S 00 and S01 The target cluster obtained by clustering the image feature points of the image to be stitched, I0, is S. 10 and S 11 , of which S 00 With S 10 Corresponding to each other, S 01 With S 11 They correspond to each other. Then, based on the target cluster S... 00 All image feature points and S 10 All image feature points can be used to perform image transformation on the images to be stitched, I0 and I1, according to the feature point matching pairs, so that the two image feature points in the feature point matching pair coincide, thereby obtaining the target cluster S. 00 With S 10 The corresponding concatenation result. In the same way, the target cluster S can be obtained. 01 With S 11 The corresponding stitching results. Then, the terminal device can determine the target image from the two stitching results.
[0066] It should be understood that in step S104, the image feature points of two planes with a disparity difference less than the difference threshold will be aggregated into a target cluster. Therefore, when stitching images, two planes with a small disparity difference will be merged and processed, while the image feature points of two planes with a disparity difference greater than or equal to the difference threshold will be divided into different target clusters. Therefore, when stitching images, two planes with a large disparity difference will be processed separately.
[0067] It should be noted that when performing image stitching based on two corresponding target clusters from different groups, the terminal device may use the same or different image stitching models.
[0068] In the embodiments of this application, two images of the target scene to be stitched and image feature points of each image to be stitched are acquired; image feature points belonging to the same plane in each image to be stitched are determined; based on the disparity values of each pair of mutually matching planes in the two images to be stitched, the disparity difference between each pair of disparity values is calculated; image feature points of two planes in the same image to be stitched whose disparity difference is less than the difference threshold are aggregated into one class to obtain the target class cluster of each image to be stitched; based on the target class cluster of each image to be stitched, the two images to be stitched are stitched together to obtain the target image. This allows the terminal device to adaptively perform image stitching based on the disparity size of the planes in the scene, separately considering each plane with a large disparity difference, realizing multi-plane stitching, avoiding ghosting or obvious stitching marks in the stitching result. At the same time, by merging the image feature points of planes with small disparity differences, it is not necessary to perform stitching processing on each plane with small disparity differences separately, thus improving the efficiency of image stitching.
[0069] Figure 2A schematic diagram of the image stitching process of this application is shown below, in conjunction with... Figure 2 The image stitching method provided in this application will be described with reference to specific implementation methods.
[0070] In some embodiments of this application, such as Figure 3 As shown, step S102 may include steps S301 to S304.
[0071] Step S301: Based on the coordinate position of each image feature point in the corresponding image to be stitched, cluster the image feature points in each image to be stitched to obtain the original clusters of each image to be stitched.
[0072] In this application, the original clusters in the two images to be stitched correspond one-to-one. It should be understood that the image stitching in this application needs to be processed separately according to the parallax of different planes. The surfaces of objects in the target scene constitute planes, and image feature points with similar coordinate positions often come from the same plane. Therefore, clustering can be performed according to the coordinate position of each image feature point.
[0073] like Figure 2 As shown in (a), the two ends of the line in the elliptical dashed box are a pair of feature point matching pairs.
[0074] The terminal device obtains image feature points and feature point matching pairs based on step S101. Then, the terminal device can cluster the image feature points in the two images to be stitched according to the coordinate position of each image feature point in the corresponding image to be stitched. The image feature points with similar coordinate positions in the same image to be stitched are clustered into the same original cluster until a preset number T of original clusters are obtained for each image to be stitched.
[0075] The preset quantity T can be set according to the check value, for example, it can be set to 5.
[0076] Based on the original clusters obtained from clustering and the feature point matching pairs between image feature points, image feature points can be used... It means that, among them, and These are image feature points. and Corresponding cluster labels It is a pair of feature point matching pairs. This indicates that the image feature points corresponding to the feature point matching pairs in the image I0 to be stitched are clustered into the original cluster with a label of 1. This indicates that the image feature points corresponding to the feature point matching pairs in the image I1 to be stitched are clustered into the original cluster with label 1.
[0077] Because the spatial distribution of image feature points in the images to be stitched (I0 and I1) differs, the coordinate positions of the feature point matching pairs in their respective images and their relative positions with other image feature points may not be consistent. This can lead to the feature point matching pairs being clustered into different original clusters in different images, for example, it may result in… The situation.
[0078] Therefore, after obtaining the original clusters of each image to be stitched, the terminal device can also calculate the similarity E between the original clusters of one image and the original clusters of the other image. If the similarity E is less than the similarity threshold, the image feature points in each image to be stitched are re-clustered to obtain new original clusters for each image, and the similarity E is redetermined based on the new original clusters until the similarity E is greater than or equal to the similarity threshold.
[0079] The similarity threshold can be set according to the actual situation.
[0080] In some embodiments of this application, the terminal device can create a T×T matrix C, where the number at each position in matrix C is a preset value, such as 0. Here, T represents the number of original clusters in an image to be stitched.
[0081] according to For feature point matching pairs, the frequency of clustering labels in each matching pair can be counted in matrix C, for example... Then we can add one to the first row and first column of matrix C. Then add one to the second row and second column of matrix C. Then add one to the second column of the third row in matrix C, and so on, until all feature point matching pairs have been traversed.
[0082] It should be understood that a reasonable clustering result should result in feature point matching pairs clustering in their corresponding original clusters. Therefore, the value at the diagonal of matrix C should be the maximum value of each row.
[0083] Based on this, the terminal device can create a new T×1 vector V to record the column number of the maximum value in each row of the matrix C. If there are duplicate values in vector V, it can be confirmed that the similarity E is less than the similarity threshold, and the image feature points in each image to be stitched can be re-clustered to obtain a new original cluster for each image to be stitched, until there are no duplicate values in vector V.
[0084] Please refer to Figure 4 , Figure 4 This diagram illustrates the clustering process of an image to be stitched together according to this application. The original clusters are as follows: Figure 4As shown in (a), different shapes represent different primitive clusters.
[0085] Step S302: Select outliers from the image feature points of each original cluster.
[0086] After obtaining reasonable original clusters, the terminal device can use the Random Sample Consensus (RANSAC) algorithm to filter the feature point matching pairs contained in each original cluster, retaining inliers and removing outliers.
[0087] In this context, interior points are the correct feature points in the original cluster; exterior points are the abnormal feature points in the original cluster.
[0088] Specifically, if the number of feature point matching pairs contained in the original cluster and its corresponding original cluster is less than or equal to a preset value, then all image feature points of the feature point matching pairs in these two original clusters can be used as interior points. The preset value can be set according to actual conditions, for example, to 8.
[0089] If the number of feature point matching pairs contained in the original cluster and its corresponding original cluster is greater than a preset value, then outliers are filtered out using the RANSAC algorithm.
[0090] Figure 4 (b) shows the screening Figure 4 (a) is a schematic diagram of the outer points and remaining inner points in the original cluster.
[0091] Step S303: Calculate the outlier rate for each cluster combination.
[0092] Each cluster combination contains any two original clusters from the same image to be stitched together.
[0093] It should be understood that feature points on the same plane can be fitted using the same homography transformation matrix, and terminal devices can determine image feature points on the same plane based on this principle.
[0094] Specifically, the terminal device can combine any two original clusters from the same image to be stitched together, resulting in a total of [number missing]. There are several combination methods, where Z represents the number of original clusters in the same image to be stitched together.
[0095] Then, the RANSAC algorithm is used to calculate the outlier rate for each combination.
[0096]
[0097] Among them, O nV is the number of outliers eliminated by the nth type of cluster combination. i and V j , which are the number of image feature points in the i-th original cluster and the j-th original cluster in the n-th cluster combination, respectively, and B = {(1,2), (1,3), ..., (Z-1,Z)} is a series of cluster combinations.
[0098] Step S304: Take all image feature points of the two original clusters in the cluster combination where the outlier rate is less than the outlier rate threshold as image feature points of the same plane in the corresponding image to be stitched.
[0099] In some embodiments of this application, if the out-of-field ratio is greater than or equal to the out-of-field ratio threshold, it indicates that the image feature points of the two original clusters in the cluster combination do not belong to the same plane, and such combination is invalid. If the out-of-field ratio is less than the out-of-field ratio threshold, it indicates that the image feature points of the two original clusters in the cluster combination belong to the same plane.
[0100] By calculating the outlier rate for each cluster combination, the terminal device can generate a connectivity graph using all valid cluster combinations. Nodes in the graph represent a cluster combination, and lines indicate that two connected cluster combinations are valid combinations. All connected components in the connectivity graph are obtained. Since the original clusters represented by the containing nodes in each connected component belong to the same plane, the number of connected components represents the number of planes contained in the scene.
[0101] The result of merging the original clusters contained in the same connected component is as follows: Figure 4 As shown in (c), points of the same shape represent image feature points in the same connected component. It can be seen that... Figure 4 In (b), the image feature points shown by the triangle and star are combined to represent image feature points belonging to the same plane.
[0102] Now, the number of planes in the target scene is known, but there may not be a large disparity between all the planes. Therefore, we need to merge the image feature points of the planes with small disparity differences through the aforementioned steps S103 and S104.
[0103] Specifically, such as Figure 5 As shown, step S103 may include steps S501 to S503.
[0104] Step S501: Obtain the coordinates of the cluster center of the image feature points of each plane in the two images to be stitched together.
[0105] Specifically, the terminal device can obtain the coordinates of the image feature points of each plane (that is, all the image feature points of the original clusters contained in each connected component mentioned above), and calculate the average value as the coordinates of the cluster center.
[0106] Step S502: Determine the disparity value of each set of mutually matching planes in the two images to be stitched, based on the coordinate positions of the cluster centers of each set of mutually matching planes in the corresponding images to be stitched.
[0107] use and These represent the coordinates of the cluster centers of each plane in the images to be stitched, I0 and I1, respectively. The terminal device can calculate the disparity value of each pair of matching planes in the two images according to the disparity formula.
[0108] Wherein, the parallax formula is:
[0109] The column represents the image width of the images to be stitched together. It is the total number of cluster centers. and Let I represent the i-th plane that matches each other in the images I0 and I1 to be stitched together.
[0110] Step S503: Calculate the disparity difference between every two disparity values based on the disparity values of each set of matching planes in the two images to be stitched.
[0111] After obtaining the disparity values of each pair of matching planes in the two images to be stitched, the terminal device can calculate the disparity difference between every two disparity values.
[0112] Accordingly, in the aforementioned step S104, the difference threshold can be... U is an empirical value that can be set according to the actual situation, for example, to 15.
[0113] In other words, if This means that the disparity between the i-th plane and the j-th plane is not large enough, so the terminal device will aggregate the image feature points of these two planes into one class to obtain the target class cluster.
[0114] In the embodiments of this application, image feature points are aggregated into original clusters based on their coordinate positions, and further clustering of these original clusters is determined by calculating the outlier rate. This identifies image feature points belonging to the same plane, accurately obtaining the number of planes in the target scene. Further clustering using the coordinate positions of the cluster centers allows planes with smaller disparity differences to be merged for processing. This eliminates the need for separate stitching for each plane with a small disparity difference in step S105 when performing image stitching based on the target clusters, improving image stitching efficiency. Conversely, planes with large disparity differences are not clustered together, requiring separate stitching for each plane in step S105 when performing image stitching based on the target clusters. This avoids ghosting or obvious stitching lines in the stitching result.
[0115] After obtaining the target cluster, such as Figure 6 As shown, in some embodiments of this application, step S105 may specifically include steps S601 to S603.
[0116] Step S601: Based on the corresponding target clusters in the two images to be stitched, perform image stitching on the two images to be stitched to obtain candidate images corresponding to each pair of corresponding target clusters.
[0117] The corresponding target clusters in two images to be stitched can be represented as F = {f1, f2, ..., f...} k}, f i This represents the i-th pair of mutually corresponding target clusters, which contains several feature point matching pairs.
[0118] Based on the corresponding target clusters F in two images to be stitched, the terminal device can estimate a local change model for each pair of corresponding target clusters in F, resulting in k candidate images. Different candidate images are obtained by performing feature point matching, alignment, and image transformation on image feature points of different target clusters.
[0119] After image stitching is completed, the terminal device can obtain the overlapping area of the two images to be stitched in each candidate image.
[0120] Please refer to Figure 2 (b), Figure 2 (b) shows different candidate images obtained by stitching together the target class clusters of two images to be stitched.
[0121] Step S602: Using any one of the candidate images as a reference, scale the other candidate images to obtain scaled candidate images.
[0122] In some embodiments of this application, the terminal device can scale other candidate images based on any one of the candidate images to make the image height and image width consistent, thereby enabling the candidate images to be compared in the same dimension.
[0123] Specifically, the terminal device can use any candidate image as the first reference image, and scale the candidate images other than the first reference image according to the image height of the first reference image until the image height of all candidate images is consistent, thus obtaining the preprocessed image corresponding to each candidate image.
[0124] Next, the terminal device can use any preprocessed image as the second reference image, and fill or delete pixels in the preprocessed images other than the second reference image based on the image width of the second reference image, until the image width of all preprocessed images is consistent, thus obtaining the scaled candidate image.
[0125] Among them, such as Figure 7 As shown, the process of filling or deleting pixels includes the following steps S701 to S705.
[0126] In step S701, the terminal device can calculate the target adjustment area of the third reference image.
[0127] The third reference image is the preprocessed image with the largest overlap between the two images to be stitched together.
[0128] Specifically, the terminal device can obtain a preset ratio value, take the center point of the overlapping area as the center point of the target adjustment area, set the height of the overlapping area to be the same as the height of the target adjustment area, and calculate the target adjustment area according to the preset ratio.
[0129] For example, the preset ratio value can be 80%.
[0130] Please refer to Figure 8 (a) The shaded area is the target adjustment area of the third reference image (the topmost image), and the black area is the overlapping area.
[0131] The target adjustment area is the area where pixels are filled or deleted; it can refer to the seamless carving area.
[0132] Step S702: Align the overlapping area of the image to be scaled with the overlapping area of the third reference image to the left, and determine the position of the right frame of the target adjustment area of the image to be scaled in the image to be scaled based on the position of the right frame of the target adjustment area in the third reference image.
[0133] The image to be scaled is a preprocessed image other than the third reference image.
[0134] Step S703: Align the overlapping area of the image to be scaled with the overlapping area of the third reference image to the right, and determine the position of the left border of the target adjustment area of the image to be scaled in the image to be scaled based on the position of the left border of the target adjustment area in the third reference image.
[0135] Please refer to Figure 8 (a) The terminal device right-aligns the overlapping area of the image to be scaled with the overlapping area of the third reference image. At this time, based on the position of the left border of the target adjustment area in the third reference image, the position of the left border of the target adjustment area in the image to be scaled can be determined, so that the left borders of the third reference image and the image to be scaled are aligned. Figure 8 The dashed line in (a) is the left border of the target adjustment area of the image to be scaled.
[0136] Please refer to Figure 8 (b) The terminal device aligns the overlapping area of the image to be scaled with the overlapping area of the third reference image to the left. At this time, based on the position of the right bounding box of the target adjustment area in the third reference image, the position of the right bounding box of the target adjustment area in the image to be scaled can be determined, so that the right bounding boxes of the third reference image and the image to be scaled are aligned. Figure 8 The dashed lines in (b) represent the left and right borders of the target adjustment area of the image to be scaled.
[0137] Step S704: Determine the target adjustment area of the image to be scaled based on the position of the right border of the target adjustment area in the image to be scaled and the position of the left border of the target adjustment area in the image to be scaled.
[0138] Based on the positions of the right and left borders within each image to be scaled, the target adjustment area for each image can be determined. That is, as... Figure 8 (c) shows the shaded area.
[0139] It should be noted that the left and right mentioned here generally refer to the X-axis direction in the image coordinate system.
[0140] Step S705: Using the image width of the second reference image as a reference, fill or delete pixels in the target adjustment area of the preprocessed image outside the second reference image until the image width of all preprocessed images is consistent, and obtain the scaled candidate image.
[0141] Specifically, the terminal device can use the Seam Carving algorithm to fill or delete pixels in the target adjustment area of the other preprocessed images, based on one of the preprocessed images, until the image width of all preprocessed images is consistent, thus obtaining the scaled candidate image.
[0142] Please refer to Figure 2 (c), Figure 2 (c) shows an intermediate image of the Seam Carving algorithm. Figure 2 (c) The pixels on the lines from top to bottom can be copied or deleted. After the pixels are filled or deleted, the image width of all preprocessed images is consistent, and the scaled candidate image is obtained.
[0143] It should be understood that this method can preserve the image content on one side of the two images to be stitched, so that the stitching results are roughly the same in the non-overlapping areas.
[0144] In the embodiments of this application, the image scaling process and pixel filling and deletion processes described above ensure that the scaled candidate images are consistent in both height and width, facilitating comparison. Using the SeamCarving algorithm to correct the position of the SeamCarving regions in the overlapping areas guarantees that the positions of the same target object in the scaled candidate images are approximately consistent, reducing positional differences caused by different image stitching models processing the same point in space. Furthermore, for non-overlapping areas, the deformation is equivalent to a similarity transformation, thus ensuring that overlapping areas are aligned as much as possible while avoiding significant stretching and deformation in non-overlapping areas.
[0145] Step S603: Determine the target image based on the scaled candidate image.
[0146] Specifically, the terminal device can calculate the data items and smoothing items corresponding to each pixel of the target image from each scaled candidate image, and based on the data items and smoothing items, form the target image from the pixels that make the Markov function reach its minimum value.
[0147] We use T to represent the final stitching result, where T(p) is the brightness value of pixel p in T, which comes from the scaled candidate image (A = {a1, a2, ... a2}). k One of them is the problem of solving the Markov energy function.
[0148] Markov functions typically consist of a data term and a smoothing term.
[0149] Specifically, the Markov function can be expressed as E(l) = ∑ p E d (l p )+α∑(p，q)∈N E s (l p , l q ).
[0150] Where α is the weight value, which can be set according to the actual situation, and the value is generally between 0 and 1. E d (l p ) represents the data item, E s (l p , l q ) represents the smoothing term.
[0151] The data items can be viewed as prior information, and the ultimate goal is to ensure that the aligned regions in the scaled candidate image A contribute to the target image T. Therefore, there are two ways to determine a at pixel p. i Is this a good choice? First, determine which plane pixel p lies on; second, determine the alignment of the stitched result in the scaled candidate image A with pixel p.
[0152] Based on this, for each data item, since the positions of the image feature points on each plane are initially known, and the image transformation parameters used by the image stitching model are known, the positions of the image feature points on the plane that each scaled candidate image is aligned with are also known.
[0153] Let H be the set of image feature points on the i-th plane. i If each image feature point is assigned a Gaussian weight G(.) according to a Gaussian function, then the confidence level of pixel p on a certain plane is... Confidence The closer an image is to a feature point on a certain plane, the higher the confidence level of belonging to that plane.
[0154] Meanwhile, the alignment of each scaled candidate image at pixel p in and B represents the images obtained by transforming I0 and I1 using the feature point matching pair on the i-th target cluster, respectively. r (p) represents the set of pixels within a distance r from pixel p. Alignment This indicates that the higher the confidence level of the scaled candidate image, the better it is aligned at pixel p.
[0155] In summary, regarding the energy loss at pixel p... Overall data items l p It is the label of which scaled candidate image the pixel p comes from.
[0156] For the smoothing term, Where p and q are two adjacent pixels in a 4-neighborhood. They are respectively the lth p The brightness value at pixel p of a scaled candidate image.
[0157] In summary, the terminal device can calculate which scaled candidate image each pixel p should come from when the Markov function E(l) reaches its minimum value, and then combine the determined pixels to form the target image.
[0158] Figure 2 (c) shows the target image.
[0159] like Figure 9 , 10 Figures 1 and 11 are schematic diagrams illustrating image stitching using different algorithms. Figure 9 , 10 In Figures 1 and 11, the dashed box in (a) represents a set of images to be stitched together; (b), (c), and (d) are target images obtained by stitching the images to be stitched together using existing image stitching methods; and (e) is a target image obtained by stitching the images to be stitched together using the image stitching method provided in the embodiments of this application. It can be seen that there are no ghosting shadows or obvious stitching line marks in the target image.
[0160] like Figure 12 As shown, 12 test participants were asked to select the best stitched image from those produced by different image stitching methods. A vote was then taken on all the stitched images. Figure 12 (a) represents the support rate of the stitched images (10 groups) of a target scene with large parallax obtained by stitching them together using different image stitching methods. Figure 12 (b) represents the support rate of the stitching results obtained by stitching together 22 sets of images of a target scene with small parallax using different image stitching methods.
[0161] As can be seen from the bar chart, for target scenes with large parallax, the image stitching method provided in this application ( Figure 12The image stitching method (OUR) obtained by this application has the highest support rate, significantly higher than the APAP (As-projective-as-possible) image stitching model, the NIS (Natural Image Stitching with the Global Similarity Prior) image stitching model, and the MR (Robust image stitching with multiple registrations) image stitching model. This indicates that the image stitching method provided in this application is indeed effective for target scenes with large parallax. For target scenes with small parallax, the image stitching method provided in this application obtains a support rate comparable to NIS, higher than APAP and SEAGULL (Seam-guided local alignment for parallax-tolerant image stitching) image stitching models. This indicates that for small parallax scenes, the image stitching method provided in this application degenerates into the less complex NIS, exhibiting scene adaptability.
[0162] It should be noted that, for the sake of simplicity, the aforementioned method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, because according to this application, some steps can be performed in other orders.
[0163] like Figure 13 The image shown is a schematic diagram of an image stitching device 1300 provided in an embodiment of this application. The image stitching device 1300 is configured on a terminal device.
[0164] Specifically, the image stitching device 1300 may include:
[0165] The acquisition unit 1301 is used to acquire two images of the target scene to be stitched together, and image feature points of each image to be stitched together; the determination unit 1302 is used to determine the image feature points belonging to the same plane in each image to be stitched together; the calculation unit 1303 is used to calculate the disparity difference between each pair of disparity values based on the disparity values of each pair of mutually matching planes in the two images to be stitched together; the clustering unit 1304 is used to aggregate the image feature points of two planes in the same image to be stitched together that have a disparity difference less than a difference threshold into one class, thereby obtaining the target cluster of each image to be stitched together; the stitching unit 1305 is used to stitch the two images to be stitched together based on the target cluster of each image to be stitched together, thereby obtaining the target image.
[0166] In some embodiments of this application, the determining unit 1302 may be specifically used to: cluster the image feature points in each image to be stitched according to the coordinate position of each image feature point in the corresponding image to be stitched, to obtain the original clusters of each image to be stitched, wherein the original clusters in two images to be stitched correspond one-to-one; filter out outliers from the image feature points of each original cluster; calculate the outlier rate of each cluster combination, wherein each cluster combination contains any two original clusters in the same image to be stitched; and take all the image feature points of the two original clusters in the cluster combination where the outlier rate is less than the outlier rate threshold as image feature points of the same plane in the corresponding image to be stitched.
[0167] In some embodiments of this application, the above-mentioned calculation unit 1303 may be specifically used to: obtain the coordinate position of the cluster center of the image feature points of each plane in the two images to be stitched; determine the disparity value of each set of mutually matching planes in the two images to be stitched based on the coordinate position of the cluster center of each set of mutually matching planes in the corresponding image to be stitched; and calculate the disparity difference between each two disparity values based on the disparity value of each set of mutually matching planes in the two images to be stitched.
[0168] In some embodiments of this application, the above-mentioned calculation unit 1303 may be specifically used to: after obtaining the original cluster of each of the images to be stitched, calculate the similarity between the original cluster of one of the two images to be stitched and the original cluster of the other image; if the similarity is less than the similarity threshold, then re-cluster the image feature points in each of the images to be stitched to obtain a new original cluster for each image to be stitched.
[0169] In some embodiments of this application, the target clusters in the two images to be stitched are one-to-one; the stitching unit 1305 can be specifically used to: stitch the two images to be stitched according to the target clusters that correspond to each other in the two images to be stitched, to obtain candidate images corresponding to each set of target clusters; scale the other candidate images based on any one of the candidate images to obtain scaled candidate images; and determine the target image based on the scaled candidate images.
[0170] In some embodiments of this application, the stitching unit 1305 can be specifically used to: scale the candidate images other than the first reference image according to the image height of the first reference image until the image height of all the candidate images is consistent, to obtain a preprocessed image corresponding to each candidate image, wherein the first reference image is any one of the candidate images; and fill or delete pixels in the preprocessed images other than the second reference image based on the image width of the second reference image until the image width of all the preprocessed images is consistent, to obtain the scaled candidate image, wherein the second reference image is any one of the preprocessed images.
[0171] In some embodiments of this application, the stitching unit 1305 can be specifically used to: calculate the target adjustment region of the third reference image, wherein the third reference image is the preprocessed image with the largest overlap between the two images to be stitched; left-align the overlap region of the image to be scaled with the overlap region of the third reference image, and determine the position of the right frame of the target adjustment region of the image to be scaled in the image to be scaled based on the position of the right frame of the target adjustment region of the third reference image in the third reference image, wherein the image to be scaled is the preprocessed image other than the third reference image; right-align the overlap region of the image to be scaled with the overlap region of the third reference image, and Based on the position of the left border of the target adjustment region of the third reference image in the third reference image, the position of the left border of the target adjustment region of the image to be scaled in the image to be scaled is determined; based on the position of the right border of the target adjustment region of the image to be scaled in the image to be scaled, and the position of the left border of the target adjustment region of the image to be scaled in the image to be scaled, the target adjustment region of the image to be scaled is determined; using the image width of the second reference image as a reference, pixels are filled or deleted in the target adjustment region of the preprocessed image outside the second reference image until the image width of all the preprocessed images is consistent, thus obtaining the scaled candidate image.
[0172] In some embodiments of this application, the stitching unit 1305 described above can be specifically used to: calculate the data item and smoothing item corresponding to each pixel of the target image from each of the scaled candidate images; and, based on the data item and the smoothing item, form the target image from the pixels that minimize the Markov function.
[0173] It should be noted that, for the sake of convenience and brevity, the specific working process of the image stitching device 1300 described above can be found in the following reference: Figures 1 to 12 The corresponding process of the method will not be described in detail here.
[0174] like Figure 14 The diagram shown is a schematic of a terminal device provided in an embodiment of this application. The terminal device 14 may include: a processor 140, a memory 141, and a computer program 142 stored in the memory 141 and executable on the processor 140, such as an image stitching program. When the processor 140 executes the computer program 142, it implements the steps in the various image stitching method embodiments described above, for example... Figure 1 Steps S101 to S105 are shown. Alternatively, when the processor 140 executes the computer program 142, it implements the functions of each module / unit in the above-described device embodiments, for example... Figure 13 The unit shown is the acquisition unit 1301, the determination unit 1302, the calculation unit 1303, the clustering unit 1304, and the splicing unit 1305.
[0175] The computer program can be divided into one or more modules / units, which are stored in the memory 141 and executed by the processor 140 to complete this application. The one or more modules / units can be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of the computer program in the terminal device.
[0176] For example, the computer program can be divided into: an acquisition unit, a determination unit, a calculation unit, a clustering unit, and a splicing unit.
[0177] The specific functions of each unit are as follows: An acquisition unit, used to acquire two images of the target scene to be stitched, and image feature points of each image to be stitched; a determination unit, used to determine the image feature points belonging to the same plane in each image to be stitched; a calculation unit, used to calculate the disparity difference between each pair of disparity values based on the disparity values of each pair of mutually matching planes in the two images to be stitched; a clustering unit, used to aggregate the image feature points of two planes in the same image to be stitched whose disparity difference is less than a threshold into one class, obtaining the target cluster of each image to be stitched; and a stitching unit, used to stitch the two images to be stitched together according to the target cluster of each image to be stitched, obtaining the target image.
[0178] The terminal device may include, but is not limited to, a processor 140 and a memory 141. Those skilled in the art will understand that... Figure 14 This is merely an example of a terminal device and does not constitute a limitation on the terminal device. It may include more or fewer components than shown, or combine certain components, or different components. For example, the terminal device may also include input / output devices, network access devices, buses, etc.
[0179] The processor 140 may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.
[0180] The memory 141 can be an internal storage unit of the terminal device, such as a hard drive or memory. The memory 141 can also be an external storage device of the terminal device, such as a plug-in hard drive, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card. Furthermore, the memory 141 can include both internal and external storage units of the terminal device. The memory 141 is used to store the computer program and other programs and data required by the terminal device. The memory 141 can also be used to temporarily store data that has been output or will be output.
[0181] It should be noted that, for the sake of convenience and brevity, the structure of the terminal device described above can also be referred to the specific description of the structure in the method embodiment, which will not be repeated here.
[0182] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.
[0183] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.
[0184] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for various specific applications, but such implementations should not be considered beyond the scope of this application.
[0185] In the embodiments provided in this application, it should be understood that the disclosed devices / terminal equipment and methods can be implemented in other ways. For example, the device / terminal equipment embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling or direct coupling or communication connection may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.
[0186] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0187] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.
[0188] If the integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electrical carrier signals and telecommunication signals.
[0189] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.
Claims
1. An image stitching method, characterized in that, include: Obtain two images of the target scene to be stitched together, and the image feature points of each image to be stitched together; Identify the image feature points belonging to the same plane in each of the images to be stitched together; Obtain the coordinates of the cluster centers of image feature points on each plane in the two images to be stitched together; Based on the coordinate positions of the cluster centers of each set of mutually matching planes in the two images to be stitched, the disparity value of each set of mutually matching planes in the two images to be stitched is determined. Based on the disparity values of each pair of mutually matching planes in the two images to be stitched, calculate the disparity difference between every two disparity values; The image feature points of two planes in the same image to be stitched together with a disparity difference less than a difference threshold are aggregated into one class to obtain the target class cluster of each image to be stitched together; The target clusters in the two images to be stitched together correspond one-to-one. Based on the corresponding target clusters in the two images to be stitched, the two images to be stitched are stitched together to obtain candidate images corresponding to each pair of corresponding target clusters; Using any one of the candidate images as a reference, the other candidate images are scaled to obtain scaled candidate images; The target image is determined based on the scaled candidate image.
2. The image stitching method as described in claim 1, characterized in that, The step of determining the image feature points belonging to the same plane in each of the images to be stitched includes: Based on the coordinate position of each image feature point in the corresponding image to be stitched, the image feature points in each image to be stitched are clustered to obtain the original cluster of each image to be stitched, wherein the original clusters in two images to be stitched correspond one-to-one; Outer points are selected from the image feature points of each of the original clusters; Calculate the out-of-point rate for each cluster combination, wherein each cluster combination contains any two of the original clusters in the same image to be stitched; All image feature points of the two original clusters in the cluster combination where the outlier rate is less than the outlier rate threshold are taken as image feature points of the same plane in the corresponding image to be stitched.
3. The image stitching method as described in claim 2, characterized in that, After obtaining the original clusters of each of the images to be stitched, the image stitching method further includes: Calculate the similarity between the original cluster of one of the two images to be stitched and the original cluster of the other image; If the similarity is less than the similarity threshold, the image feature points in each of the images to be stitched together are re-clustered to obtain a new original cluster for each of the images to be stitched together.
4. The image stitching method as described in claim 1, characterized in that, The step of scaling the other candidate images based on any one of the candidate images to obtain scaled candidate images includes: Based on the image height of the first reference image, the candidate images other than the first reference image are scaled until the image height of all the candidate images is consistent, thereby obtaining a preprocessed image corresponding to each candidate image, wherein the first reference image is any one of the candidate images; Using the image width of the second reference image as a reference, the preprocessed images other than the second reference image are padded or deleted with pixels until the image width of all the preprocessed images is consistent, thus obtaining the scaled candidate image, wherein the second reference image is any one of the preprocessed images.
5. The image stitching method as described in claim 4, characterized in that, The process of filling or deleting pixels in the preprocessed images other than the second reference image, based on the image width of the second reference image, until all the preprocessed images have the same image width, to obtain the scaled candidate image, includes: Calculate the target adjustment region of the third reference image, wherein the third reference image is the preprocessed image with the largest overlap between the two images to be stitched; Align the overlapping region of the image to be scaled with the overlapping region of the third reference image to the left, and determine the position of the right frame of the target adjustment region of the image to be scaled in the image to be scaled based on the position of the right frame of the target adjustment region of the third reference image in the third reference image. The image to be scaled is the preprocessed image other than the third reference image. Align the overlapping area of the image to be scaled with the overlapping area of the third reference image to the right, and determine the position of the left border of the target adjustment area of the image to be scaled in the image to be scaled based on the position of the left border of the target adjustment area of the third reference image in the third reference image. The target adjustment area of the image to be scaled is determined based on the position of the right border of the target adjustment area in the image to be scaled and the position of the left border of the target adjustment area in the image to be scaled. Using the image width of the second reference image as a reference, pixels are filled or deleted in the target adjustment area of the preprocessed image outside the second reference image until the image width of all the preprocessed images is consistent, thus obtaining the scaled candidate image.
6. The image stitching method as described in claim 1, characterized in that, Determining the target image from the scaled candidate images includes: Calculate the data item and smoothing item corresponding to each pixel of the target image from each scaled candidate image; Based on the data items and the smoothing items, the target image is composed of the pixels that minimize the Markov function.
7. An image stitching device, characterized in that, include: The acquisition unit is used to acquire two images of the target scene to be stitched together, and image feature points of each of the images to be stitched together; A determining unit is used to determine the image feature points belonging to the same plane in each of the images to be stitched together; The calculation unit is used to obtain the coordinate positions of the cluster centers of the image feature points of each plane in the two images to be stitched; determine the disparity value of each set of mutually matching planes in the two images to be stitched based on the coordinate positions of the cluster centers of each set of mutually matching planes in the corresponding image to be stitched; and calculate the disparity difference between each pair of disparity values based on the disparity values of each pair of mutually matching planes in the two images to be stitched. A clustering unit is used to aggregate the image feature points of two planes in the same image to be stitched together into one class, so as to obtain the target cluster of each image to be stitched together; The target clusters in the two images to be stitched together correspond one-to-one. The stitching unit is used to stitch two images to be stitched together according to the target clusters that correspond to each other in the two images to be stitched together, to obtain candidate images corresponding to each set of target clusters; to scale the other candidate images based on any one of the candidate images, to obtain scaled candidate images; and to determine the target image based on the scaled candidate images.
8. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the image stitching method as described in any one of claims 1 to 6.
9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the image stitching method as described in any one of claims 1 to 6.