Image processing method and device, equipment, chip and storage medium

Abstract

The embodiment of the invention discloses an image processing method and device, equipment, a chip and a storage medium. The method comprises the steps of obtaining a pair of first images including the same object; determining a first homography matrix between the pair of first images based on the first error threshold, wherein the error of the first homography matrix is within the first error threshold; under the condition that the first homography matrix meets the first condition, a second homography matrix between the pair of first images is determined based on a second error threshold value, the error of the second homography matrix is within the second error threshold value, and the second error threshold value is larger than the first error threshold value; under the condition that the second homography matrix meets the first condition, a second error threshold value is output, the second error threshold value is used for determining a target homography matrix between a pair of second images comprising the same object, and the error of the target homography matrix is within the second error threshold value; the target homography matrix is used for processing the pair of second images.

Description

Technical Field

[0001] The present application relates to the field of signal processing technology, and in particular to an image processing method, device, equipment, chip and storage medium. Background Art

[0002] After the dual-camera device captures images, the captured images need to be processed (such as image fusion, distortion correction, etc.). In this process, it is usually necessary to calculate the transformation relationship between the images captured by different cameras (such as calculating the homography matrix). The calculation process of the homography matrix is ​​a process that gradually converges the results through multiple iterative optimizations. As the number of iterations increases, the running time of the algorithm will increase linearly.

[0003] With the development of dual-camera devices, the requirements for real-time image processing are becoming increasingly higher. Therefore, it is necessary to compress the running time of image processing algorithms. Summary of the invention

[0004] The present application at least provides an image processing method, apparatus, device, chip and storage medium.

[0005] The technical solution of this application is implemented as follows:

[0006] In a first aspect, an embodiment of the present application provides a method for image processing, the method comprising: acquiring a pair of first images including the same object; determining a first homography matrix between the pair of first images based on a first error threshold, the error of the first homography matrix being within the first error threshold; determining a second homography matrix between the pair of first images based on a second error threshold when the first homography matrix satisfies a first condition, the error of the second homography matrix being within a second error threshold, wherein the second error threshold is greater than the first error threshold; outputting a second error threshold when the second homography matrix satisfies the first condition, the second error threshold being used to determine a target homography matrix between a pair of second images including the same object, the error of the target homography matrix being within the second error threshold; the target homography matrix is ​​used to process the pair of second images.

[0007] In a second aspect, an embodiment of the present application provides an image processing device, which includes: an acquisition module, used to acquire a pair of first images including the same object; a first determination module, used to determine a first homography matrix between the pair of first images based on a first error threshold, the error of the first homography matrix is ​​within the first error threshold; a second determination module, used to determine a second homography matrix between the pair of first images based on a second error threshold when the first homography matrix satisfies a first condition, the error of the second homography matrix is ​​within the second error threshold, wherein the second error threshold is greater than the first error threshold; a first output module, used to output a second error threshold when the second homography matrix satisfies the first condition, the second error threshold is used to determine a target homography matrix between a pair of second images including the same object, the error of the target homography matrix is ​​within the second error threshold; the target homography matrix is ​​used to process the pair of second images.

[0008] In a third aspect, an embodiment of the present application provides a communication device, which includes a memory and a processor; wherein the memory is used to store computer-executable instructions; the processor is connected to the memory and is used to implement the method described in the first aspect by executing the computer-executable instructions.

[0009] In a fourth aspect, an embodiment of the present application provides a chip, which includes: a processor, configured to call and run a computer program from a memory, so that a device equipped with the chip executes the method described in the first aspect.

[0010] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by at least one processor, it implements the method described in the first aspect.

[0011] In an embodiment of the present application, a first homography matrix between a pair of first images including the same object can be determined based on a first error threshold, so that the error of the first homography matrix is ​​within the first error threshold; when the first homography matrix satisfies the first condition, the error threshold can be increased from the first error threshold to the second error threshold, and a second homography matrix between the pair of first images can be determined based on the second error threshold, so that the error of the second homography matrix is ​​within the second error threshold; further, when the second homography matrix satisfies the first condition, a second error threshold can be output, and the second error threshold can be used to determine a target homography matrix between a pair of second images including the same object, so that the error of the target homography matrix is ​​less than the second error threshold; the target homography matrix can be used to process the pair of second images. By increasing the allowable error threshold, the method can reduce the number of iterations of the algorithm in the process of determining the homography matrix, thereby effectively reducing the running time of the algorithm.

[0012] It should be understood that the above general description and the following detailed description are merely exemplary and explanatory, and are not intended to limit the technical solutions of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present application and are used together with the specification to illustrate the technical solution of the present application.

[0014] Figure 1 is a schematic diagram of a pair of images containing 4 matching point pairs;

[0015] Figure 2 This is a schematic diagram of an example of using the RANSAC algorithm to fit a straight line;

[0016] Figure 3 This is a schematic diagram of an image before using the RANSAC algorithm to eliminate outliers;

[0017] Figure 4 This is a schematic diagram of an example of using the RANSAC algorithm to eliminate outliers in an image;

[0018] Figure 5 A flowchart of a method for calculating eigenvalues ​​and eigenvectors using Jacobi iteration method;

[0019] Figure 6 This is a schematic diagram of an example of the traversal order when traversing a symmetric matrix row by row;

[0020] Figure 7 A schematic diagram of an example of finding the maximum element of each row of a symmetric matrix;

[0021] Figure 8 A schematic diagram of an example of finding the maximum element of each column of a symmetric matrix;

[0022] Fig. 9 A flowchart of an image processing method provided in an embodiment of the present application;

[0023] Fig.10 This is another example of using the RANSAC algorithm to fit a straight line;

[0024] Fig.11 This is a schematic diagram of an example of feature point matching error;

[0025] Fig.12 This is a schematic diagram of an example of a method for simulating a gradual reduction in overall error during an iteration according to an embodiment of the present application;

[0026] Fig.13This is a schematic diagram of an example of simulating the distribution of errors introduced by random point selection according to an embodiment of the present application;

[0027] Fig.14 This is a schematic diagram of an example of the overall error change of the simulation algorithm during the iteration process in the embodiment of the present application;

[0028] Fig.15 for Fig.14 The distribution diagram of U corresponding to 500 points when the number of iterations in is 120;

[0029] Fig.16 for Fig.14 The distribution diagram of U corresponding to 500 points when the number of iterations in is 10;

[0030] Fig.17 Another schematic diagram of an embodiment of the present application for simulating the overall error change of the algorithm during the iteration process;

[0031] Fig.18 This is another schematic diagram of an embodiment of the present application for simulating the overall error change of the algorithm during the iteration process;

[0032] Fig.19 A schematic diagram of an example of the relationship between the adjustment step size of the Homo matrix and the number of iterations in the iteration process of the simulation algorithm in an embodiment of the present application;

[0033] Fig. 20 A schematic diagram of an example of the relationship between the calculation result and the number of iterations in the iteration process of the simulation algorithm in an embodiment of the present application;

[0034] Fig.21 A schematic diagram of an example of the relationship between the running time and the number of iterations of a simulation algorithm in an iterative process according to an embodiment of the present application;

[0035] Fig. 22 This is an illustration of the experimental results of calculating the Homo matrix based on the RANSAC algorithm. Figure 1 ;

[0036] Fig.23 This is an illustration of the experimental results of calculating the Homo matrix based on the RANSAC algorithm. Figure 2 ;

[0037] Fig.24 A schematic diagram of the structure of an image processing device provided in an embodiment of the present application;

[0038] Fig.25 A schematic diagram of a hardware entity of a communication device in an embodiment of the present application. DETAILED DESCRIPTION

[0039] In order to enable a more detailed understanding of the features and technical contents of the embodiments of the present application, the implementation of the embodiments of the present application is described in detail below in conjunction with the accompanying drawings. The attached drawings are for reference only and are not used to limit the embodiments of the present application.

[0040] Unless otherwise defined, all technical and scientific terms used in the embodiments of the present application have the same meaning as those commonly understood by those skilled in the art of the present application. The terms used in the embodiments of the present application are only for the purpose of describing the embodiments of the present application and are not intended to limit the present application.

[0041] In the following description, reference is made to "some embodiments", which describe a subset of all possible embodiments, but it is understood that "some embodiments" may be the same subset or different subsets of all possible embodiments, and may be combined with each other without conflict. It should also be noted that the terms "first\second\third" involved in the embodiments of the present application are only used to distinguish similar objects and do not represent a specific ordering of the objects. It is understandable that "first\second\third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0042] After the dual-camera device captures images, the captured images need to be processed (such as image fusion, distortion correction, etc.). In this process, it is usually necessary to calculate the transformation relationship between the images captured by different cameras (such as calculating the homography matrix). The calculation process of the homography matrix is ​​a process that gradually converges the results through multiple iterative optimizations. As the number of iterations increases, the running time of the algorithm will increase linearly.

[0043] With the development of dual-camera devices, the requirements for real-time image processing are becoming increasingly higher. Therefore, it is necessary to compress the running time of image processing algorithms.

[0044] In view of this, the embodiments of the present application provide a method, device, equipment, chip and storage medium for image processing. In the method, a first homography matrix between a pair of first images including the same object can be determined based on a first error threshold, so that the error of the first homography matrix is ​​within the first error threshold. When the first homography matrix satisfies the first condition, the error threshold allowed in the process of determining the homography matrix can be increased, for example, it can be increased from the first error threshold to the second error threshold. Further, the second homography matrix between the pair of first images can be determined based on the second error threshold, so that the error of the second homography matrix is ​​within the second error threshold. If the second homography matrix can still meet the first condition at this time, it can be considered that increasing the error threshold to the second error threshold has no effect or little effect on the result, so the second error threshold can be used for determining the target homography matrix, wherein the target homography matrix is ​​the homography matrix between a pair of second images including the same object. By increasing the error threshold allowed in the process of determining the homography matrix, the method can reduce the number of iterations of the algorithm in the process of determining the homography matrix, thereby effectively reducing the running time of the algorithm.

[0045] To facilitate understanding of the technical solutions of the embodiments of the present application, the following is an explanation of the relevant technologies / terms of the embodiments of the present application. The following related technologies can be combined arbitrarily with the technical solutions of the embodiments of the present application as optional solutions, and they all belong to the protection scope of the embodiments of the present application.

[0046] 1. Homography matrix (Homo matrix for short)

[0047] The Homo matrix can be used to describe the transformation relationship between two planes. It is a 3×3 homogeneous matrix. The Homo matrix can map points in one image to corresponding points in another image.

[0048] For example, the Homo matrix H can be expressed as formula (1):

[0049]

[0050] The H mapping relationship between two images can be expressed as formula (2):

[0051]

[0052] Where i (i = 1, 2, 3, 4) represents the sequence number of the point pair. In the i-th matching point pair, the coordinates of one of the points can be expressed as (x i ,y i ), the coordinates of another point can be expressed as (x i ′,y i′). To obtain H between two images, at least 4 point pairs with the same corresponding positions (i.e. 4 matching point pairs) are required.

[0053] Figure 1 is a schematic diagram of a pair of images containing 4 matching point pairs. Figure 1 As shown, Figure 1 The four points circled in (a) can be compared with Figure 1 The four points circled in (b) correspond to each other. H can be expressed as Figure 1 The transformation relationship from Figure (a) to Figure (b).

[0054] By expanding formula (2), we can obtain the following formula (3):

[0055]

[0056] After that, we can compare the first two equations to the third equation and set the right side of the equation equal to 0, thus obtaining equation (4):

[0057]

[0058] Expand formula (4) into matrix A i In the form of h=0, the following formulas (5) and (6) can be obtained:

[0059]

[0060] h=[h1 h2 h3 h4 h5 h6 h7 h8 h9] T (6);

[0061] If all four point pairs are taken into account, and A i h=0, then the following formula (7) can be obtained:

[0062]

[0063] The 8×9 matrix on the left side of the equal sign can be recorded as matrix A, for example. Further, the singular value decomposition (SVD) algorithm can be used to convert matrix A into a symmetric matrix A', and the matrix H can be obtained by calculating the eigenvalues ​​and eigenvectors of the symmetric matrix A'. Exemplarily, the symmetric matrix A' can be converted into an (approximate) diagonal matrix (that is, the elements outside the main diagonal are (approximately) 0) by the Jacobi iteration method, so that the (approximate) eigenvalues ​​and (approximate) eigenvectors of the symmetric matrix A' can be determined based on the (approximate) diagonal matrix. After obtaining the (approximate) eigenvalues ​​of the symmetric matrix A', the minimum (approximate) eigenvalue of all (approximate) eigenvalues ​​of the symmetric matrix A' and the (approximate) eigenvector corresponding to the minimum (approximate) eigenvalue can be determined. Therefore, h1 to h8 in the matrix H can be composed of elements in the (approximate) eigenvector corresponding to the minimum (approximate) eigenvalue of the symmetric matrix A', and h9=1.

[0064] 2. Random Sample Consensus (RANSAC) algorithm

[0065] RANSAC is an iterative algorithm that estimates the parameters of a mathematical model from a set of observation data containing "outliers" through iteration. RANSAC is also an "outlier" detection algorithm, an uncertain algorithm with a certain probability of obtaining a reasonable result. In order to increase the probability of obtaining a reasonable result, the number of iterations needs to be increased. When the number of iterations increases, the probability of obtaining a reasonable result also increases.

[0066] The algorithm flow of RANSAC may include the following steps 11 to 15.

[0067] Step 11: Randomly sample K points, where K is the minimum number of points required to solve the model parameters.

[0068] Step 12: Estimate model parameters using K points.

[0069] Step 13: Calculate the distance between the remaining points and the estimated model. If the distance is less than the threshold, it is an inlier point, and count the number of inliers.

[0070] Step 14: Repeat steps 11 to 13 M times and retain the largest number of inliers.

[0071] Step 15: Re-estimate the model using all inliers.

[0072] Figure 2 The step of fitting a straight line using the RANSAC algorithm may include steps 21 to 25.

[0073] Step 21: Randomly select K = 2 points. Figure 2 (a) in the figure, where the solid dots represent the selected points.

[0074] Step 22: Fit a straight line using the two selected points. Figure 2 Figure (b) in .

[0075] Step 23: Count the number of inliers. Figure 2 (c) in the figure, where the solid circles represent the interior points.

[0076] Step 24: Repeat steps 21 to 23 for M times and keep the largest number of inliers. Figure 2 (d) in the figure, where the solid circles represent the interior points.

[0077] Step 25: Refit the line using all interior points. Figure 2 Figure (e) in .

[0078] As mentioned above, to obtain the Homo matrix H between two images, at least 4 matching point pairs are required. As an implementation method, the 4 matching point pairs can be determined using the RANSAC algorithm.

[0079] As an example, an implementation process of using the RANSAC algorithm to determine four point pairs for calculating the Homo matrix H is introduced below. The implementation process may include steps 31 to 35.

[0080] Step 31: Randomly sample 4 point pairs from the set S.

[0081] The set S may include at least 4 point pairs, which may be obtained by, for example, matching feature points of a pair of images using a feature point matching algorithm. The feature point matching algorithm may be any algorithm capable of matching feature points between two images, such as a scale-invariant feature transform (SIFT) algorithm, a speeded up robust features (SURF) algorithm, etc., which is not limited in the present embodiment of the application.

[0082] Step 32: Estimate the Homo matrix using 4 randomly sampled point pairs.

[0083] Step 33: Match the point pairs in the set S based on the Homo matrix and determine the internal points.

[0084] For example, for each point pair in the set S, the position of the other point can be estimated based on the Homo matrix and the position of one of the points in the point pair, and the estimation error can be calculated based on the estimated position of the other point and the true position of the other point. If the estimation error is within the threshold t, the point in the point pair can be determined as an inlier, otherwise it is an outlier. By traversing each point pair in the set S, an inlier set S consisting of all inliers can be obtained. j .

[0085] One possible situation is that if the set S j If the ratio of the number of internal points in the set S to the number of all points in the set S exceeds the threshold T, step 34 can be executed:

[0086] Step 34: Eliminate (remove) external points.

[0087] Another possible situation is that if the set S j If the ratio of the number of internal points in the set S to the number of all points in the set S does not exceed the threshold T, steps 31 to 33 may be repeated, and the number of repetitions shall not exceed M times.

[0088] After the number of repetitions reaches M, a set of interior points S with the largest number of interior points can be selected. j , and eliminate the internal point set S j The corresponding external points, that is, eliminating the internal point set S j Points other than the points in .

[0089] As an example, Figure 3 This is a schematic diagram of an image before using the RANSAC algorithm to eliminate outliers; Figure 4 This is a schematic diagram of an example of using the RANSAC algorithm to eliminate outliers in an image.

[0090] The RANSAC algorithm has the following differences compared to the least squares method: the least squares method can adapt to all points including external points as much as possible. On the contrary, the RANSAC algorithm can obtain a model calculated only with internal points, and the obtained model has a sufficiently high reasonable probability. However, the RANSAC algorithm cannot guarantee that the result is correct. In order to ensure that the algorithm has a sufficiently high reasonable probability, the parameters (parameter configuration) of the algorithm must be carefully selected. Experimental verification shows that for a data set containing 80% error, the effect of the RANSAC algorithm is far better than the least squares method.

[0091] 3. Jacobi Iteration Method

[0092] Figure 5 The flow chart of a method for calculating eigenvalues ​​and eigenvectors using Jacobi iteration method is shown. The method may include the following steps:

[0093] S501, initializing the eigenvector matrix to a diagonal matrix V.

[0094] The elements on the main diagonal of the diagonal matrix V are 1, and the other elements are 0.

[0095] S502, finding the maximum element on the off-diagonal line of the matrix S.

[0096] Among them, matrix S is the symmetric matrix of the eigenvalues ​​and eigenvectors to be calculated, and the maximum element refers to the element with the largest absolute value. In this step, it is necessary to find the element with the largest absolute value among the non-main diagonal elements of matrix S. Assume that the element with the largest absolute value is recorded as S ij , where i and j indicate that the element is located in the i-th row and j-th column of the matrix S.

[0097] S503, calculating the rotation angle θ.

[0098] First, tan(2θ) is calculated using the following formula (8):

[0099]

[0100] According to tan(2θ), cosθ, sinθ, and S can be calculated. ij The corresponding rotation matrix J(i,j,θ). The rotation matrix J(i,j,θ) can be defined as follows:

[0101]

[0102] Where s = sinθ, c = cosθ. For simplicity, J(i,j,θ) is denoted as J in the following text.

[0103] S504, updating the row and column where the maximum element is located.

[0104] The updated matrix S (for example, denoted as S′) can be expressed as the product of the following three items: the transpose of the rotation matrix J, the matrix S, and the rotation matrix J. That is, S′=J T ×S×J. From this, we can get the calculation formula of S′, see formula (10):

[0105]

[0106] Wherein, s=sinθ, c=cosθ. According to the formula, the i-th row element, j-th row element, i-th column element and j-th column element of S are updated. In addition, the updated eigenvector matrix (e.g., denoted as V′) can be expressed as the product of the current eigenvector matrix V and the rotation matrix J, that is, V′=VJ.

[0107] S505, determining whether the maximum element on the off-diagonal line of the matrix S is less than a threshold.

[0108] If the maximum value of the non-main diagonal elements of S' is less than a given threshold value e, the calculation is stopped; otherwise, let S = S', V = V', and repeat S502 to S505. When the calculation is stopped, the elements on the main diagonal of S' can be determined as the (approximate) eigenvalues ​​of the initial matrix S, and the column vector of V' can be determined as the (approximate) eigenvector of the initial matrix S.

[0109] S506, end.

[0110] In a possible manner, in the above process of calculating eigenvalues ​​and eigenvectors, the calculation of the inverse angle can be avoided by using a formula for changing the double angle.

[0111] It should be noted that the symmetric matrix mentioned many times in the embodiments of the present application refers to a real symmetric matrix unless there is any ambiguity; the diagonal line mentioned many times in the present application refers to the main diagonal line unless there is any ambiguity.

[0112] It should also be noted that the embodiment of the present application does not limit the method for finding the maximum element (such as the element with the largest absolute value) on the non-diagonal line of the matrix S. Exemplarily, finding the maximum element on the non-diagonal line of the matrix S can be achieved, for example, by the following method 1 and method 2:

[0113] Method 1: Traverse all elements on the off-diagonal line of matrix S, find the maximum element, and record the row and column where the maximum element is located. For example, when traversing matrix S, you can traverse all elements on the off-diagonal line of matrix S by row or by column. Figure 6 As shown in FIG. 1 , it shows a schematic diagram of an example of the traversal order when traversing a symmetric matrix row by row. Figure 6 In the figure, circles represent elements in a symmetric matrix, and the shaded elements represent the elements on the diagonal of the symmetric matrix. Since the elements in the upper triangle and the lower triangle of a symmetric matrix are equal, when searching for the largest element on the off-diagonal of the symmetric matrix, we only need to traverse the elements in the upper triangle or lower triangle (excluding the elements on the diagonal), for example, Figure 6 Only the elements in the upper triangle can be traversed. The advantage of this method is that the largest element on the off-diagonal line of the matrix S can be accurately found, but the disadvantage is that more comparisons are required.

[0114] Method 2: Find and record the position of the maximum element of each row and column on the off-diagonal line of the matrix S, and then compare the maximum elements of all rows and columns to find the maximum element, which is the final maximum element on the off-diagonal line of the matrix S. In subsequent iterations, since only the row and column where the maximum element is located are updated each time, after each iteration, it is only necessary to re-determine the maximum element on the row and column, and there is no need to re-determine the maximum elements on other rows and columns.

[0115] As an example, Figure 7 This is a schematic diagram of an example of finding the maximum element of each row of a symmetric matrix. Figure 7 In the figure, the shaded elements are the maximum elements of each row in the upper triangular position (excluding the diagonal). Figure 8 This is a schematic diagram of an example of finding the maximum element in each column of a symmetric matrix. Figure 8 In the example, the shaded elements are the maximum elements of each column in the upper triangle (excluding the diagonal). Figure 7 and Figure 8 Compare all the shaded elements in the array to determine the largest element.

[0116] Compared with method 1, this method is conducive to compressing the traversal range when the matrix dimension is large, thereby saving the amount of calculation. However, the disadvantage of this method is that the maximum element finally determined may not be the actual maximum element on the off-diagonal line of the matrix S, or in other words, the maximum element found by this method may be inaccurate, so it cannot be guaranteed that the actual maximum element on the off-diagonal line can be eliminated in each iteration. In addition, when the matrix dimension is relatively small, the advantage of this method is not obvious.

[0117] In the embodiments of the present application, the matrix dimensions involved are relatively small. For example, in the process of calculating the Homo matrix, it is usually necessary to calculate the eigenvalues ​​and eigenvectors of a 9×9 symmetric matrix. Therefore, the above-mentioned method 1 can be used to find the maximum element on the non-diagonal line of the matrix S to ensure that each iteration can update the row and column where the actual maximum element on the non-diagonal line of the matrix S is located, thereby improving the convergence speed of the result.

[0118] The above briefly describes the related technologies / terms involved in the embodiments of the present application, which will not be repeated in the following embodiments.

[0119] To facilitate understanding of the technical solutions of the embodiments of the present application, the technical solutions of the present application are described in detail below through specific embodiments. The above related technologies can be combined arbitrarily with the technical solutions of the embodiments of the present application as optional solutions, and they all belong to the protection scope of the embodiments of the present application. The embodiments of the present application include at least part of the following contents.

[0120] Fig. 9 An image processing method provided by an embodiment of the present application is shown, and the method may include:

[0121] S901, acquiring a pair of first images including the same object.

[0122] The pair of first images of the same object may be, for example, images captured from different perspectives of the same scene, or images captured from different perspectives of the same object. The pair of first images may be captured by a dual-camera device, for example.

[0123] Assuming that the pair of first images includes image #1 and image #2, then, for a certain feature point in image #1 (for example, recorded as feature point #1), a feature point (for example, recorded as feature point #2) corresponding to the feature point (feature point #1) can be found in image #2. The feature point #1 and the feature point #2 can form a matching feature point pair (hereinafter referred to as a point pair), or in other words, each feature point pair can be composed of two matching feature points. Generally speaking, image #1 and image #2 may contain multiple matching point pairs, and the multiple matching point pairs can be obtained by matching the feature points of the pair of first images using a feature point matching algorithm, for example. Among them, the feature point matching algorithm can be any algorithm that can achieve feature point matching between two images, such as a SIFT algorithm, a SURF algorithm, etc., which is not limited in the embodiments of the present application.

[0124] S902: Determine a first homography matrix between the pair of first images based on a first error threshold, wherein an error of the first homography matrix is ​​within the first error threshold.

[0125] In this step, the first homography matrix between the pair of first images can be determined based on the first error threshold, and the error of the first homography matrix is ​​within the first error threshold. In other words, in the process of determining the first homography matrix, the first error threshold can be used as a basis for determining the first homography matrix, so that the error of the determined first homography matrix is ​​within the first error threshold, or in other words, the error of the first homography matrix is ​​controlled within the first error threshold.

[0126] Among them, the first error threshold can be a numerical value or a numerical range, which is not limited in the embodiments of the present application. When the first error threshold is a numerical value, the error of the first homography matrix is ​​within the first error threshold, which can also be understood as the error of the first homography matrix is ​​less than the first error threshold; when the first error threshold is a numerical range, the error of the first homography matrix is ​​within the first error threshold, which can also be understood as the error of the first homography matrix is ​​within the numerical interval defined by the numerical range.

[0127] Since the calculation process of the homography matrix is ​​a process in which the result gradually converges through multiple iterative optimizations, therefore, in the process of calculating the homography matrix between the pair of first images, if the error of the homography matrix obtained at the Kth iteration (K is a positive integer) is within the first error threshold, the iteration can be stopped, and the homography matrix obtained at the Kth iteration is used as the determined first homography matrix.

[0128] As an implementation method, the error of the homography matrix (such as the first homography matrix, the second homography matrix and the target homography matrix) in the embodiment of the present application can be determined based on the symmetric matrix associated with the homography matrix. Among them, the symmetric matrix associated with the homography matrix can also be understood as a symmetric matrix used to calculate the homography matrix.

[0129] For example, in formula (7), the 8×9 matrix A can be converted into a symmetric matrix A', and the homography matrix H can be obtained by calculating the eigenvalues ​​and eigenvectors of the symmetric matrix A'. For example, the symmetric matrix A' can be converted into an (approximate) diagonal matrix by Jacobi iteration, so that the (approximate) eigenvalues ​​and (approximate) eigenvectors of the symmetric matrix A' can be determined based on the (approximate) diagonal matrix. In this example, the symmetric matrix A' is the symmetric matrix associated with the homography matrix H. Among them, the error of the homography matrix H can be expressed as: the difference between the element with the largest absolute value on the non-main diagonal line in the (approximate) diagonal matrix (obtained by transforming the symmetric matrix A') and 0.

[0130] S903, when the first homography matrix satisfies the first condition, determining the second homography matrix between the pair of first images based on a second error threshold, wherein the error of the second homography matrix is ​​within the second error threshold, wherein the second error threshold is greater than the first error threshold.

[0131] In the case where the first homography matrix satisfies the first condition, the second homography matrix between the pair of first images can be determined based on the second error threshold, the error of the second homography matrix is ​​within the second error threshold, and the second error threshold is greater than the first error threshold. In other words, in the case where the first homography matrix satisfies the first condition, the error threshold allowed in the process of determining the homography matrix can be increased, for example, the error threshold can be increased from the first error threshold to the second error threshold. Further, the second homography matrix between the pair of first images can be determined based on the second error threshold, so that the error of the determined second homography matrix is ​​within the second error threshold, or in other words, the error of the second homography matrix is ​​controlled within the second error threshold.

[0132] Among them, the second error threshold can be a numerical value or a numerical range, which is not limited in the embodiments of the present application. When the second error threshold is a numerical value, the error of the second homography matrix is ​​within the second error threshold, which can also be understood as the error of the second homography matrix is ​​less than the second error threshold; when the second error threshold is a numerical range, the error of the second homography matrix is ​​within the second error threshold, which can also be understood as the error of the second homography matrix is ​​within the numerical interval defined by the numerical range.

[0133] Since the calculation process of the homography matrix is ​​a process in which the result gradually converges through multiple iterative optimizations, therefore, in the process of calculating the homography matrix between the pair of first images, if the error of the homography matrix obtained at the Lth (L is a positive integer) iteration is within the second error threshold, the iteration can be stopped, and the homography matrix obtained at the Lth iteration is used as the determined second homography matrix.

[0134] In some embodiments, before determining the second homography matrix between the pair of first images based on the second error threshold, the method may further include: determining whether the first homography matrix satisfies the first condition. In one possible manner, if the first homography matrix can make most (e.g., more than 90%) of the feature points in the pair of first images be determined as interior points, it can be determined that the first homography matrix satisfies the first condition.

[0135] As an example, determining whether the first homography matrix satisfies the first condition may be achieved, for example, by following the steps 9031) to 9033):

[0136] 9031) Match at least one matching point pair in the pair of first images based on the first homography matrix to obtain at least one first matching error; wherein each point pair corresponds to a first matching error.

[0137] For example, for each matching point pair in the pair of first images, the position of the other point can be estimated based on the first homography matrix and the position of one of the points in the point pair, and the difference between the estimated position of the other point and the true position of the other point is calculated, so that the calculated difference is used as the first matching error corresponding to the point pair.

[0138] 9032) For each point pair, when the first matching error corresponding to the point pair is less than a third error threshold, a point in the point pair is determined as a first inner point.

[0139] For example, assuming that the at least one matching point pair is composed of point pair #1, point pair #2, point pair #3 and point pair #4, and the first matching error corresponding to point pair #1, point pair #2 and point pair #3 is less than the third error threshold, and the first matching error corresponding to point pair #4 is greater than the third error threshold, then the points in point pair #1, point pair #2 and point pair #3 can be determined as inner points (for example, recorded as the first inner points), and the points in point pair #4 can be determined as outer points (for example, recorded as the first outer points). That is, the number of first inner points is 3×2=6, and the number of first outer points is 2. It should be noted that if the first matching error corresponding to a point pair is equal to the third error threshold, then, in one implementation, the points in the point pair can be considered as inner points, and in another implementation, the points in the first point pair can also be considered as outer points, which is not limited in the embodiments of the present application.

[0140] 9033) When the number of first internal points is greater than the first number, determine that the first homography matrix satisfies the first condition.

[0141] Exemplarily, the first number can be determined based on the total number of all inner points and outer points, or in other words, the first number can be determined based on the total number of points in all point pairs. For example, assuming that the at least one matching point pair is composed of point pair #1, point pair #2, point pair #3, and point pair #4, then the total number of points in all point pairs is 4×2=8. As an implementation, the first number can be, for example, 90% of the total number of all inner points and outer points (the total number of points in all point pairs). That is, when the number of the first inner points is greater than 90% of the total number of all inner points and outer points, it can be determined that the first homography matrix satisfies the first condition.

[0142] It should be understood that the specific values ​​of the number of point pairs and the first number in the embodiment of the present application are merely exemplary and should not constitute any limitation on the implementation process of the embodiment of the present application.

[0143] According to the method of this embodiment, when the first homography matrix can make most (such as more than 90%) of the feature points in the acquired pair of first images be determined as inliers, it can be determined that the first homography matrix satisfies the first condition. Since the larger the number (or proportion) of inliers, the larger the number of feature points that the homography matrix can adapt to, that is, the more reliable the result obtained, therefore, by associating the first condition with the number (or proportion) of inliers, it is helpful to improve the reliability of the determined homography matrix (such as the first homography matrix).

[0144] S904, when the second homography matrix satisfies the first condition, output a second error threshold, the second error threshold is used to determine a target homography matrix between a pair of second images including the same object, and the error of the target homography matrix is ​​within the second error threshold; the target homography matrix is ​​used to process the pair of second images.

[0145] The pair of second images of the same object may be, for example, images captured from different perspectives of the same scene, or images captured from different perspectives of the same object. The pair of second images may be captured by a dual-camera device, for example.

[0146] In one possible manner, when the second homography matrix satisfies the first condition, a second error threshold can be output, and the second error threshold can be used to determine the target homography matrix between the pair of second images, so that the error of the determined target homography matrix is ​​within the second error threshold, or in other words, the error of the target homography matrix is ​​controlled within the second error threshold.

[0147] It can be understood that if the second homography matrix can still meet the first condition, it means that increasing the error threshold from the first error threshold to the second error threshold has no effect or little effect on the result. Therefore, in the subsequent process of determining the target homography matrix, the second error threshold can be used as the basis for determining the target homography matrix, so that the error of the determined target homography matrix is ​​within the second error threshold. In this way, by increasing the error threshold allowed in the process of determining the homography matrix, the number of iterations of the algorithm in the process of determining the homography matrix can be reduced, thereby effectively reducing the running time of the algorithm.

[0148] It should be understood that, as another implementation, when the second homography matrix satisfies the first condition, the allowable error threshold can be further increased. For example, the error threshold can be increased from the second error threshold to the fourth error threshold, and the homography matrix between the pair of first images (for example, recorded as the third homography matrix) is calculated again based on the fourth error threshold. If the third homography matrix can still meet the first condition, it means that after increasing the error threshold to the fourth error threshold, there is still no effect or little effect on the result. In this case, the above steps can be repeated to continue to increase the allowable error threshold until the homography matrix obtained by a certain calculation does not meet the first condition. At this time, the error threshold based on the homography matrix that meets the first condition obtained for the last time can be determined as the final error threshold and output. The above process can be completed, for example, in the training stage of the algorithm. In the actual application stage, that is, in the subsequent process of determining the target homography matrix, the final determined error threshold can be used as the basis for determining the target homography matrix.

[0149] In some embodiments, the method may further include: when the second homography matrix does not satisfy the first condition, outputting a first error threshold, the first error threshold being used to determine the target homography matrix between the pair of second images, and the error of the target homography matrix being within the first error threshold.

[0150] It is understandable that if the second homography matrix does not satisfy the first condition, it means that after increasing the error threshold from the first error threshold to the second error threshold, the desired result cannot be achieved, or the result is greatly affected. For example, the second homography matrix cannot make most (such as more than 90%) of the feature points in the acquired pair of first images be determined as inliers. In this case, the error threshold before the increase (the first error threshold) can be output, and the error threshold before the increase can be used as the basis for determining the target homography matrix to ensure the reliability of the result.

[0151] In some embodiments, before outputting the second error threshold, the method may further include: determining whether the second homography matrix satisfies the first condition. In one possible manner, if the second homography matrix can make most (e.g., more than 90%) of the feature points in the acquired pair of first images be determined as inliers, then it can be determined that the second homography matrix satisfies the first condition.

[0152] As an example, determining whether the second homography matrix satisfies the first condition may be achieved, for example, by following the steps 9041) to 9043):

[0153] 9041) Match at least one matching point pair in the pair of first images based on the second homography matrix to obtain at least one second matching error; wherein each point pair corresponds to a second matching error.

[0154] 9042) For each point pair, when the second matching error corresponding to the point pair is less than a third error threshold, a point in the point pair is determined as a second inner point.

[0155] 9043) When the number of the second inner points is greater than the first number, determine whether the second homography matrix satisfies the first condition.

[0156] Among them, the implementation method of steps 9041) to 9043) can refer to the aforementioned steps 9031) to 9033) and will not be repeated here.

[0157] In some embodiments, determining whether the second homography matrix satisfies the first condition can also be achieved in the following way: when the first homography matrix satisfies the first condition and the absolute value of the difference between the corresponding elements of the second homography matrix and the first homography matrix is ​​less than a preset value, determining that the second homography matrix satisfies the first condition.

[0158] Among them, the corresponding elements of the second homography matrix and the first homography matrix can also be understood as the elements in the same position of the second homography matrix and the first homography matrix, such as the elements in the i-th row and j-th column of the second homography matrix and the elements in the i-th row and j-th column of the first homography matrix.

[0159] As an implementation method, if the absolute values ​​of the differences between all corresponding elements of the second homography matrix and the first homography matrix are less than a preset value, it can be considered that the absolute value of the difference between the corresponding elements of the second homography matrix and the first homography matrix is ​​less than the preset value; as another implementation method, if the absolute value of the difference between at least one corresponding element of the second homography matrix and the first homography matrix is ​​less than a preset value, it can be considered that the absolute value of the difference between the corresponding elements of the second homography matrix and the first homography matrix is ​​less than the preset value.

[0160] It can be understood that if the absolute value of the difference between the corresponding elements of the second homography matrix and the first homography matrix is ​​less than the preset value, it means that the difference between the homography matrices determined before and after the error threshold is increased is small, which means that the effect of increasing the error threshold on the result is small. Therefore, when it is determined that the first homography matrix satisfies the first condition (for example, it can be determined by steps 9031) to 9033), if the absolute value of the difference between the corresponding elements of the second homography matrix and the first homography matrix is ​​less than the preset value, then it can also be considered that the second homography matrix satisfies the first condition. Furthermore, in the subsequent process of determining the target homography matrix, the second error threshold can be used as the basis for determining the target homography matrix, so that the error of the determined target homography matrix is ​​within the second error threshold.

[0161] In some embodiments, the duration used in the process of determining the first homography matrix is ​​the first duration, and the duration used in the process of determining the second homography matrix is ​​the second duration. The method may further include steps 9044) and 9045):

[0162] 9044) Determine a second quantity based on a difference between the first duration and the second duration, and the second quantity is positively correlated with the difference between the first duration and the second duration.

[0163] 9045) Perform the first operation a second number of times to obtain a target point pair set for determining a target homography matrix; the first operation includes: randomly determining a candidate point pair set from a plurality of matching point pairs in the pair of second images; wherein the target point pair set is determined from all candidate point pair sets.

[0164] Exemplarily, the multiple matching point pairs in the pair of second images can be obtained by, for example, matching the feature points of the pair of second images using a feature point matching algorithm. A candidate point pair set can include, for example, 4 candidate point pairs, that is, each time the first operation is performed, 4 candidate point pairs can be determined.

[0165] For example, assuming that the second number is 10, the first operation can be performed 10 times. Each time the first operation is performed, a candidate point pair set can be obtained, so after performing the first operation 10 times, 10 candidate point pair sets can be obtained. Further, a candidate point pair set with the highest reliability can be determined from the 10 candidate point pair sets as the target point pair set. As an implementation method, a target homography matrix can be calculated using each candidate point pair set, and the number of inliers that can be obtained based on each target homography matrix (or the number of feature points that can be determined as inliers based on each target homography matrix) can be counted respectively. Among them, the more inliers that can be obtained based on the target homography matrix, the higher the reliability of the candidate point pair set used when calculating the target homography matrix. Generally speaking, the larger the second number, the more reliable the target point pair set obtained. In other words, as the second number increases, the probability of obtaining a reliable target point pair set also increases.

[0166] It can be understood that, since the second homography matrix is ​​determined based on the second error threshold, the first homography matrix is ​​determined based on the first error threshold, and the second error threshold is greater than the first error threshold, that is, the allowable error threshold is increased (or the requirement for error is relaxed) in the process of calculating the second homography matrix, therefore, the second duration is usually less than the first duration. In this way, the difference between the first duration and the second duration can be considered as the running time of the algorithm saved after relaxing the requirement for error.

[0167] In this embodiment, the difference between the first duration and the second duration may be positively correlated with the second quantity. That is, the larger the difference between the first duration and the second duration (i.e., the longer the running time of the algorithm saved), the larger the second quantity (i.e., more time is invested in determining the set of target point pairs), so that the obtained set of target point pairs can be made more reliable. As an implementation method, for example, the corresponding relationship between the difference between the first duration and the second duration and the second quantity can be preset, and the larger the difference between the first duration and the second duration, the larger the second quantity. In this way, after obtaining the difference between the first duration and the second duration, the second quantity to be adopted can be determined based on the corresponding relationship.

[0168] According to the method of this embodiment, if the saved algorithm running time is longer, more time can be invested in determining the target point pair set, which is beneficial to improve the reliability of the target point pair set used to determine the target homography matrix when the overall algorithm running time remains unchanged or changes slightly, thereby improving the accuracy of the obtained target homography matrix. Among them, the target point pair set may include 4 point pairs, for example. Determining the target homography matrix based on the target point pair set can be achieved by solving formula (7), for example.

[0169] In some embodiments, the method may further include: determining a second error threshold based on the error of the first homography matrix and the first error threshold; wherein the difference between the second error threshold and the first error threshold is negatively correlated with the error of the first homography matrix.

[0170] Among them, the difference between the second error threshold and the first error threshold can be used to characterize the change (adjustment) of the second error threshold relative to the first error threshold, that is, it can be used to characterize the adjustment amplitude of the error threshold when the allowable error threshold is increased. That is to say, in this embodiment, when the error of the first homography matrix is ​​small (or when the accuracy of the first homography matrix is ​​high), the adjustment amplitude of the error threshold can be appropriately increased; when the error of the first homography matrix is ​​large (or when the accuracy of the first homography matrix is ​​low), the adjustment amplitude of the error threshold can be appropriately reduced.

[0171] As an implementation method, for example, the corresponding relationship between the difference between the second error threshold and the first error threshold and the error of the first homography matrix can be preset, and the larger difference between the second error threshold and the first error threshold can correspond to the smaller error of the first homography matrix. In this way, after obtaining the error of the first homography matrix, the difference between the corresponding second error threshold and the first error threshold can be determined based on the error of the first homography matrix, and then when the first error threshold is known, the second error threshold can be calculated based on the difference between the second error threshold and the first error threshold.

[0172] In some embodiments, the method may further include steps 9046) and 9047):

[0173] 9046) For at least one pair of second images, perform feature point matching on each pair of second images to obtain at least one symmetric matrix; wherein each symmetric matrix corresponds to a pair of second images.

[0174] Taking one of the at least one pair of second images as an example, by performing feature point matching on the pair of second images, a plurality of matching point pairs in the pair of second images can be obtained, wherein the plurality of matching point pairs can be obtained, for example, by performing feature point matching on the pair of second images using a feature point matching algorithm. Further, 4 target point pairs can be determined from the plurality of matching point pairs, and the 4 target point pairs can form a target point pair set, which can be used to determine the target homography matrix between the pair of second images. The target point pair set can be determined, for example, according to the aforementioned step 9045).

[0175] After determining the four target point pairs, an 8×9 matrix (for example, denoted as matrix A) can be obtained according to formula (7). Then, the SVD algorithm can be used to convert the matrix A into a symmetric matrix A', which is the symmetric matrix corresponding to the pair of second images.

[0176] 9047) Jacobi iteration method is used to update each symmetric matrix to obtain a target homography matrix between a pair of second images corresponding to each symmetric matrix.

[0177] Exemplarily, each symmetric matrix is ​​updated using the Jacobi iteration method to obtain a target homography matrix between a pair of second images corresponding to each symmetric matrix, including steps a) and b):

[0178] a) performing the second operation at least once to obtain the eigenvalues ​​and eigenvectors of each symmetric matrix; wherein the second operation includes: determining the maximum element on the off-diagonal line of each symmetric matrix, and updating the elements of the row and column of each maximum element in the corresponding symmetric matrix based on the rotation matrix corresponding to each maximum element.

[0179] As an implementation manner, the maximum element may be, for example, an element with the largest absolute value.

[0180] The rotation matrix corresponding to a certain maximum element refers to the rotation matrix of the maximum element in the corresponding (located) symmetric matrix. The rotation matrix corresponding to each maximum element can be determined by formula (9). Afterwards, the elements on the row and column of each maximum element in the corresponding symmetric matrix can be updated by formula (10).

[0181] By performing the second operation at least once, each symmetric matrix can be converted into a (approximate) diagonal matrix, and thus, the (approximate) eigenvalues ​​and (approximate) eigenvectors of the corresponding symmetric matrix can be determined based on each (approximate) diagonal matrix.

[0182] In one possible manner, based on the rotation matrix corresponding to each maximum element, the elements of the row and column of each maximum element in the corresponding symmetric matrix are updated, including: extracting the elements of the row and column of each maximum element in the corresponding symmetric matrix to obtain an element set; based on the rotation matrix corresponding to each maximum element, updating the elements of the row and column of each maximum element in the corresponding symmetric matrix in the element set; based on the position of each element before the update in the corresponding symmetric matrix, saving each updated element to the corresponding position in the corresponding symmetric matrix.

[0183] As an implementation method, the scheme of the embodiment of the present application can be executed on a digital signal processor (Digital Signal Processor, DSP). In this case, the gather instruction can be used to extract the elements on the row and column where each maximum element is located in the corresponding symmetric matrix to obtain an element set. Afterwards, the update of the elements on the row and column where each maximum element is located in the corresponding symmetric matrix can be completed in the element set. In this way, by extracting and concentrating the elements to be updated and then updating them uniformly, the parallel processing function of the DSP can be effectively utilized, and the elements on the rows and columns where the maximum elements on the non-diagonal lines of multiple symmetric matrices are located can be updated at the same time to improve the execution efficiency of the algorithm. After each update is completed, the scatter instruction can be used to save the updated elements back to the corresponding position in the original symmetric matrix. For example, assuming that the element d in the element set is extracted from the 2nd row and 3rd column of the symmetric matrix D, and the updated value of the element d is d', then after the update is completed, d' can be saved back to the 2nd row and 3rd column of the symmetric matrix D. After d' is saved back to the 2nd row and 3rd column of the symmetric matrix D, the element d at the original position can be overwritten.

[0184] b) determining a target homography matrix between a pair of second images corresponding to each symmetric matrix based on the eigenvalues ​​and eigenvectors of each symmetric matrix.

[0185] Taking a certain symmetric matrix as an example, after obtaining the (approximate) eigenvalue of the symmetric matrix, the minimum (approximate) eigenvalue of all (approximate) eigenvalues ​​of the symmetric matrix and the (approximate) eigenvector corresponding to the minimum (approximate) eigenvalue can be determined. Thus, the first 8 elements in the target homography matrix between a pair of second images corresponding to the symmetric matrix can be composed of the elements in the (approximate) eigenvector corresponding to the minimum (approximate) eigenvalue of the symmetric matrix, and the value of the 9th element is 1.

[0186] In some embodiments, the method may further include: determining a target homography matrix between a pair of second images including the same object based on a second error threshold. Since the second error threshold is greater than the first error threshold, compared with a solution of determining the target homography matrix based on the first error threshold, this solution can reduce the number of iterations of the algorithm in the process of determining the target homography matrix, thereby effectively reducing the running time of the algorithm.

[0187] In some embodiments, the method may further include: processing the pair of second images based on the target homography matrix. As an example, after obtaining the target homography matrix, the pair of second images may be processed based on the target homography matrix, but not limited to, at least one of the following processes: image fusion, image cropping, image display, and distortion correction.

[0188] The technical principles involved in the technical solutions of the embodiments of the present application are briefly analyzed and explained below.

[0189] The RANSAC algorithm uses randomly selected points to fit the function, and the process of obtaining the Homo matrix is ​​the process of fitting the function in an iterative manner. Therefore, there are two errors:

[0190] The source of the first error is that the random input point itself has a certain error relative to the exact value, and this error is caused by the input point itself. Fig.10 For example, the points to be fitted are Fig.10 As shown in Figure (a), it is assumed that the straight line l1 is the exact solution. Due to the existence of errors, the actual points are mainly distributed on both sides of the straight line l1. When two points are randomly selected for straight line fitting (see Fig.10 As shown in Figure (b), the fitted straight line l2 has a certain error compared to the exact solution l1.

[0191] The principle is the same as that of straight line fitting. When the RANSAC algorithm is used to determine the input data of the Homo matrix, since the input data is 4 point pairs, and the mapping relationship of these point pairs is obtained by matching feature points, there are errors in the matching of feature points. Therefore, when the RANSAC algorithm is used to determine the input data of the Homo matrix, there are errors in the input data itself. Fig.11 This is a schematic diagram of an example of feature point matching error. Fig.11 As shown in the figure, when matching feature points, Fig.11 Point A and point B in are determined as a matching point pair.

[0192] The second error comes from the calculation process of the Homo matrix. Since the process of obtaining the Homo matrix uses iterative calculation, the calculation stops when the iteration reaches a certain level. Therefore, the calculated result will also have a certain deviation from the actual result.

[0193] The second error can be simulated using the following formula (11):

[0194]

[0195] The value of x starts from 1, and after each iteration, x increases by 1. At the same time, rand is randomly updated, and rand randomly takes a value of -1 or 1. Formula (11) can be used to simulate the process of gradually reducing the error during the iteration process.

[0196] Fig.12 This is a schematic diagram of an example of a method for simulating the gradual reduction of errors during iterations in an embodiment of the present application. Fig.12 As shown, the error in the iterative process gradually converges.

[0197] In order to simulate the overall change of the above two errors as the iteration proceeds, it is assumed that the first error is U1 and the second error is U2, so that the overall error can be expressed as U=U1+U2. Among them, U1 conforms to the normal distribution (such as Fig.13 That is, the model generated by most of the selected points fluctuates around the exact solution, and the model generated by a small number of selected points deviates far from the exact solution of the model.

[0198] To simulate the size of U after iterating to a certain extent when both U1 and U2 exist, 500 points can be used for simulation. The overall error after superposition of U1 and U2 is as follows: Fig.14 shown. Fig.15 for Fig.14 The distribution of U corresponding to 500 points when the number of iterations in is 120. Fig.16 for Fig.14 The distribution of U corresponding to 500 points when the number of iterations in is 10. Fig.15 and Fig.16 The horizontal axis represents U, and the vertical axis represents the number of points.

[0199] pass Fig.14 It can be seen that due to the existence of U1, as the number of iterations increases, U cannot converge to 0, but a value close to 0. In addition, as the number of iterations increases, the adjustment amount (change amount) of U brought about by each iteration becomes smaller and smaller. Therefore, the solution of increasing the number of iterations to improve the calculation accuracy is not feasible, especially on the mobile side. Due to the particularity of the RANSAC algorithm, the input data (the selected 4 point pairs) often have certain errors (simulated by U1). Therefore, it is understandable that when there are errors in the input data itself, it is not meaningful to increase the number of iterations to improve the calculation accuracy. Therefore, it is possible to consider increasing the allowable error to reduce the number of iterations, thereby compressing the execution time of the algorithm within the allowable range.

[0200] The experimental results show that the total running time consumed by the iteration is related to the number of random point selections in the RANSAC algorithm and the number of iterations in the process of calculating the Homo matrix, and they are linearly increasing. When the total running time consumed by the iteration process is fixed, the results obtained by adjusting the number of random point selections and the number of iterations in the process of calculating the Homo matrix are as follows: Fig.17 and Fig.18 shown.

[0201] exist Fig.17 In , the number of random point selections n = 50, and the number of iterations is 120; Fig.18 In the example, the number of random point selections n=80 and the number of iterations is 75. Fig.17 and Fig.18 It can be seen that by increasing the number of random point selections and reducing the number of iterations, better results can be achieved while keeping the running time unchanged.

[0202] From the above reasoning, it can be seen that the quality of the calculation result of the Homo matrix mainly depends on the accuracy of the input point pairs for calculating the Homo matrix (that is, it is hoped that the 4 input point pairs are all high-precision internal points).

[0203] The calculation process of the Homo matrix is ​​a gradual convergence process. For the calculation of a single Homo matrix, the calculation of eigenvalues ​​and eigenvectors using the Jacobi iteration algorithm takes up the main computing time for calculating the Homo matrix. During the iteration process, the adjustment step size of the calculated Homo matrix gradually decreases, and as the number of iterations increases, the running time of the algorithm increases linearly. Among them, the adjustment step size can also be understood as the adjustment amount (change amount) between the Homo matrices calculated twice adjacently.

[0204] Fig.19 FIG. 1 is a schematic diagram of an example of the relationship between the adjustment step size of the Homo matrix and the number of iterations in the iterative process of the simulation algorithm in the embodiment of the present application. Fig.19 As shown in Figure 2, as the number of iterations increases, the adjustment step size of the Homo matrix gradually decreases.

[0205] Fig. 20 Schematic diagram of the relationship between the calculation result (the value in the Homo matrix) and the number of iterations of the simulation algorithm in the iteration process in the embodiment of the present application. Fig. 20 As shown in Figure 3, as the number of iterations increases, the calculated Homo matrix gradually approaches the exact solution.

[0206] Fig.21 FIG. 1 is a schematic diagram of an example of the relationship between the running time and the number of iterations of a simulation algorithm in an iterative process according to an embodiment of the present application. Fig.21As shown, as the number of iterations increases, the running time of the algorithm increases linearly.

[0207] Through Figures 19 to 21 From the analysis, we can see that when calculating the Homo matrix, the following situation may occur: in 100 iterations, the last 10% of the iterations have less than 0.01% impact on the result, but take up 10% of the iteration time. In this case, we can consider increasing the allowable error to reduce the overall time of the algorithm. Furthermore, this saved time can be used for new point selection calculations (or more time can be invested in random point selection), thereby increasing the probability of selecting reliable internal points to achieve the overall optimal effect.

[0208] To further illustrate, Fig. 22 The experimental result 1 of calculating the Homo matrix based on the RANSAC algorithm is shown; Fig.23 The experimental result 2 of calculating the Homo matrix based on the RANSAC algorithm is shown. The number of iterations for calculating the Homo matrix is ​​110 times in total. Fig. 22 The values ​​in the Homo matrix calculated after 90 iterations are shown. Fig.23 The values ​​in the Homo matrix calculated when the iteration is 110 are shown. It can be seen that as the iteration proceeds, the adjustment amplitude (variation) of the calculation result becomes smaller and smaller, and the difference (variation) between the intermediate result of the iteration to 90 times and the final result of the iteration to 110 times is less than 0.00001.

[0209] According to experience, the Homo matrix has little effect on the final error within this range of variation. Therefore, it is possible to relax the requirements for iterative error to a certain extent, thereby compressing the algorithm running time by reducing the number of iterations within the controllable error range.

[0210] The solution of the embodiment of the present application can achieve the following functions:

[0211] 1) During the training phase, the adjustment range of the calculation results during the calculation iteration process is calculated. The adjustment range of the calculation results can also be understood as the adjustment amount (change amount) between the Home matrices calculated at different numbers of iterations. For example, assuming that in the 100 iterations, the last 10% of the iterations have less than 0.01% impact on the results, but take up 10% of the iteration time. In this case, the allowable error can be increased, thereby reducing the overall running time of the algorithm. This saved time can be used for new point selection calculations, thereby increasing the probability of selecting reliable internal points to achieve the overall optimal effect. For example, in Fig. 22 and 23In the example, as the iteration proceeds, the adjustment range of the calculation result becomes smaller and smaller. Taking a total of 110 iterations as an example, the difference between the intermediate result of iteration 90 and the final result of iteration 110 is less than 0.00001. Therefore, the number of iterations can be compressed. For example, the iteration error can be amplified according to the empirical value to reduce the number of iterations.

[0212] During the operation, after each adjustment of the allowable error, the accuracy of the Homo matrix generated under the allowable error can be checked. If the accuracy is high, the adjustment range of the allowable error can be increased. If the accuracy is low, the adjustment range of the allowable error can be reduced, thereby compressing the running time of the algorithm by reducing the number of iterations within a controllable error range.

[0213] 2) For multiple symmetric matrices, since the positions of the maximum elements on the off-diagonal lines of different symmetric matrices may be different, due to the limitation of the device function, it is impossible to uniformly update the rows and columns where the maximum elements on the off-diagonal lines of multiple symmetric matrices are located by the Jacobi iteration method at the same time. In order to utilize the parallel processing function of the device (such as DSP), it is usually only possible to uniformly update the rows and columns where the relatively large elements on the off-diagonal lines of all symmetric matrices are located, rather than updating the rows and columns where the maximum elements are located, which will reduce the convergence speed. To this end, in an embodiment of the present application, a layer of conversion between the coordinates and position numbers of the upper triangular elements can be added. For example, the gather instruction can be used to extract and concentrate the data at different positions in each symmetric matrix (such as the elements on the rows and columns where the maximum elements on the off-diagonal lines of each symmetric matrix are located) and then operate. In this process, the coordinates of the element in the symmetric matrix can be converted into a position number, wherein the position number can be used to characterize the absolute position of the element in all the extracted data. After the calculation is completed, the scatter instruction can be used to save the extracted data back to the corresponding position in the symmetric matrix. In this way, it is possible to perform iterative updates according to the maximum element on the off-diagonal line each time during the iteration process, thereby improving the convergence speed. In addition, the solution can use the parallel processing function of DSP to perform parallel update operations on multiple matrices (such as 32 matrices), thereby solving the eigenvalues ​​and eigenvectors of multiple symmetric matrices at the same time.

[0214] The scheme of the embodiment of the present application has the following characteristics: 1) By increasing the permissible error of iteration in the process of calculating the Homo matrix, the total number of iterations is effectively reduced. If the reliability of the Homo matrix calculated based on the adjusted error is high (such as satisfying the first condition), the permissible error range can be further increased, and so on, so as to achieve a balance between calculation accuracy and execution performance. 2) In combination with the calculation principle of the Homo matrix, the maximum position of the non-diagonal elements under different matrices is saved through the gather / scatter instruction, so that the maximum element positions on the non-diagonal lines of different matrices are different and parallel operations can be achieved. Solve the problem of different operation positions of different matrices during vectorized operations.

[0215] The preferred embodiments of the present application are described in detail above in conjunction with the accompanying drawings. However, the present application is not limited to the specific details in the above embodiments. Within the technical concept of the present application, the technical solution of the present application can be subjected to a variety of simple modifications, and these simple modifications all belong to the protection scope of the present application. For example, the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. In order to avoid unnecessary repetition, the present application will not further explain various possible combinations. For another example, the various different embodiments of the present application can also be arbitrarily combined, as long as they do not violate the idea of ​​the present application, they should also be regarded as the contents disclosed in the present application. For another example, under the premise of no conflict, the various embodiments and / or the technical features in the various embodiments described in the present application can be arbitrarily combined with the prior art, and the technical solution obtained after the combination should also fall within the protection scope of the present application.

[0216] Based on the foregoing embodiments, an embodiment of the present application provides an image processing device, which includes the modules included and can be implemented by a processor in the device; of course, it can also be implemented by a specific logic circuit; in the implementation process, the processor can be a central processing unit (CPU), a microprocessor (MPU), a digital signal processor (DSP) or a field programmable gate array (FPGA), etc.

[0217] Fig.24 FIG. 2 shows the composition structure of an image processing device 2400 provided in an embodiment of the present application. Fig.24 As shown, the apparatus 2400 may include:

[0218] An acquisition module 2401 is used to acquire a pair of first images including the same object; a first determination module 2402 is used to determine a first homography matrix between a pair of first images based on a first error threshold, wherein the error of the first homography matrix is ​​within the first error threshold; a second determination module 2403 is used to determine a second homography matrix between a pair of first images based on a second error threshold when the first homography matrix satisfies a first condition, wherein the error of the second homography matrix is ​​within the second error threshold, wherein the second error threshold is greater than the first error threshold; an output module 2404 is used to output a second error threshold when the second homography matrix satisfies the first condition, wherein the second error threshold is used to determine a target homography matrix between a pair of second images including the same object, wherein the error of the target homography matrix is ​​within the second error threshold; and the target homography matrix is ​​used to process a pair of second images.

[0219] In some embodiments, the device 2400 also includes: a first matching module, used to match at least one matching point pair in a pair of first images based on the first homography matrix to obtain at least one first matching error; wherein each point pair corresponds to a first matching error; a third determination module, used to determine, for each point pair, a point in the point pair as a first inlier point when the first matching error corresponding to the point pair is less than a third error threshold; and a fourth determination module, used to determine that the first homography matrix satisfies the first condition when the number of first inliers is greater than the first number.

[0220] In some embodiments, the device 2400 also includes: a second matching module, used to match at least one matching point pair in a pair of first images based on a second homography matrix to obtain at least one second matching error; wherein each point pair corresponds to a second matching error; a fifth determination module, used to determine, for each point pair, a point in the point pair as a second inlier point when the second matching error corresponding to the point pair is less than a third error threshold; and a sixth determination module, used to determine that the second homography matrix satisfies the first condition when the number of second inliers is greater than the first number.

[0221] In some embodiments, the device 2400 also includes: a seventh determination module, used to determine that the second homography matrix satisfies the first condition when the first homography matrix satisfies the first condition and the absolute value of the difference between the corresponding elements of the second homography matrix and the first homography matrix is ​​less than a preset value.

[0222] In some embodiments, the duration used for determining the first homography matrix is ​​the first duration, and the duration used for determining the second homography matrix is ​​the second duration, and the device 2400 also includes: an eighth determination module, used to determine the second number based on the difference between the first duration and the second duration, and the second number is positively correlated with the difference between the first duration and the second duration; an execution module, used to execute the first operation a second number of times to obtain a target point pair set for determining the target homography matrix; the first operation includes: randomly determining a candidate point pair set from multiple matching point pairs in a pair of second images; wherein the target point pair set is determined from all candidate point pair sets.

[0223] In some embodiments, the device 2400 also includes: a ninth determination module, used to determine a second error threshold based on the error of the first homography matrix and the first error threshold; wherein the difference between the second error threshold and the first error threshold is negatively correlated with the error of the first homography matrix.

[0224] In some embodiments, the device 2400 also includes: a second output module, used to output a first error threshold when the second homography matrix does not satisfy the first condition, the first error threshold is used to determine the target homography matrix between a pair of second images, and the error of the target homography matrix is ​​within the first error threshold.

[0225] In some embodiments, the device 2400 also includes: a feature point matching module, which is used to perform feature point matching on each pair of second images for at least one pair of second images to obtain at least one symmetric matrix; wherein each symmetric matrix corresponds to a pair of second images; and an updating module, which is used to update each symmetric matrix using the Jacobi iteration method to obtain a target homography matrix between a pair of second images corresponding to each symmetric matrix.

[0226] In some embodiments, the update module includes: an execution submodule, used to perform at least one second operation to obtain the eigenvalues ​​and eigenvectors of each symmetric matrix; wherein the second operation includes: determining the maximum element on the non-diagonal line of each symmetric matrix, and based on the rotation matrix corresponding to each maximum element, updating the elements on the rows and columns of each maximum element in the corresponding symmetric matrix; a determination submodule, used to determine the target homography matrix between a pair of second images corresponding to each symmetric matrix based on the eigenvalues ​​and eigenvectors of each symmetric matrix.

[0227] In some embodiments, the execution submodule is specifically used to: extract the elements on the row and column where each maximum element is located in the corresponding symmetric matrix to obtain an element set; based on the rotation matrix corresponding to each maximum element, update the elements on the row and column where each maximum element is located in the corresponding symmetric matrix in the element set; based on the position of each element before the update in the corresponding symmetric matrix, save each updated element to the corresponding position in the corresponding symmetric matrix.

[0228] In some embodiments, the apparatus 2400 further includes: a tenth determination module, configured to determine a target homography matrix between a pair of second images based on a second error threshold.

[0229] The description of the above device embodiment is similar to the description of the above method embodiment, and has similar beneficial effects as the method embodiment. In some embodiments, the functions or modules included in the device provided in the embodiment of the present application can be used to execute the method described in the above method embodiment. For technical details not disclosed in the device embodiment of the present application, please refer to the description of the method embodiment of the present application for understanding.

[0230] It should be noted that in the embodiment of the present application, if the above method is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the relevant technology can be embodied in the form of a software product, which is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a U disk, a mobile hard disk, a read-only memory (ROM), a disk or an optical disk. In this way, the embodiment of the present application is not limited to any specific hardware, software or firmware, or any combination of hardware, software, and firmware.

[0231] An embodiment of the present application also provides a communication device, including a memory and a processor, wherein the memory stores a computer program that can be executed on the processor, and when the processor executes the program, some or all of the steps in the above method are implemented.

[0232] The embodiment of the present application further provides a chip, which includes: a processor, which is used to call and run a computer program from a memory, so that a device equipped with the chip executes some or all of the steps in the above method.

[0233] The embodiment of the present application also provides another chip, which includes a processor and a communication interface, and the processor reads instructions stored in the memory through the communication interface to implement some or all of the steps in the above method. In some embodiments, as an implementation method, the chip also includes a memory, in which a computer program or instruction is stored, and the processor is used to execute the computer program or instruction stored in the memory. When the computer program or instruction is executed, the processor is used to execute some or all of the steps in the above method.

[0234] The embodiment of the present application also provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, some or all of the steps in the above method are implemented. The computer-readable storage medium can be transient or non-transient.

[0235] An embodiment of the present application also provides a computer program, including a computer-readable code. When the computer-readable code runs in a device, a processor in the device executes some or all of the steps for implementing the above method.

[0236] The present application also provides a computer program product, which includes a non-transitory computer-readable storage medium storing a computer program, and when the computer program is read and executed by a computer, some or all of the steps in the above method are implemented. The computer program product can be implemented specifically by hardware, software, or a combination thereof. In some embodiments, the computer program product is specifically embodied as a computer storage medium, and in other embodiments, the computer program product is specifically embodied as a software product, such as a software development kit (SDK), etc.

[0237] It should be noted here that the description of the various embodiments above tends to emphasize the differences between the various embodiments, and the same or similar aspects can be referenced to each other. The description of the above device, storage medium, computer program and computer program product embodiments is similar to the description of the above method embodiment, and has similar beneficial effects as the method embodiment. For technical details not disclosed in the embodiments of the device, storage medium, computer program and computer program product of this application, please refer to the description of the method embodiment of this application for understanding.

[0238] Fig.25 A schematic diagram of a hardware entity of a communication device in an embodiment of the present application. Fig.25 The communication device 2500 shown includes a processor 2510, which can call and run a computer program from a memory to implement the method in the embodiment of the present application.

[0239] In some embodiments, Fig.25 As shown, the communication device 2500 may further include a memory 2520. The processor 2510 may call and run a computer program from the memory 2520 to implement the method in the embodiment of the present application. The memory 2520 may be a separate device independent of the processor 2510, or may be integrated in the processor 2510.

[0240] In some embodiments, Fig.25 As shown, the communication device 2500 may further include a transceiver 2530, and the processor 2510 may control the transceiver 2530 to communicate with other devices, specifically, to send information or data to other devices, or to receive information or data sent by other devices. The transceiver 2530 may include a transmitter and a receiver. The transceiver 2530 may further include an antenna, and the number of antennas may be one or more.

[0241] It should be understood that "one embodiment", "an embodiment" or "some embodiments" mentioned throughout the specification means that specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present application. Therefore, "in one embodiment", "in an embodiment" or "in some embodiments" appearing throughout the specification may not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. It should be understood that in various embodiments of the present application, the size of the serial number of each step / process mentioned above does not mean the order of execution, and the execution order of each step / process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present application. The serial numbers of the embodiments of the present application mentioned above are for description only and do not represent the advantages and disadvantages of the embodiments.

[0242] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0243] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units or modules is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored, or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0244] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units; they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0245] In addition, all functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may be a separate unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0246] A person skilled in the art can understand that all or part of the steps of implementing the above method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiment; and the aforementioned storage medium includes: mobile storage devices, read-only memories (ROM), magnetic disks or optical disks, etc., various media that can store program codes.

[0247] Alternatively, if the above-mentioned integrated unit of the present application is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can essentially or in other words, the part that contributes to the relevant technology can be embodied in the form of a software product, which is stored in a storage medium and includes a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROMs, magnetic disks, or optical disks.

[0248] The above is only an implementation method of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application.

Claims

1. A method for image processing, characterized in that: The method comprises: Acquiring a pair of first images including the same object; determining a first homography matrix between the pair of first images based on a first error threshold, wherein an error of the first homography matrix is ​​within the first error threshold; In a case where the first homography matrix satisfies a first condition, determining a second homography matrix between the pair of first images based on a second error threshold, an error of the second homography matrix being within the second error threshold, wherein the second error threshold is greater than the first error threshold; When the second homography matrix satisfies the first condition, the second error threshold is output, and the second error threshold is used to determine a target homography matrix between a pair of second images including the same object, and the error of the target homography matrix is ​​within the second error threshold; the target homography matrix is ​​used to process the pair of second images.

2. The method according to claim 1, characterized in that The method further comprises: Matching at least one matching point pair in the pair of first images based on the first homography matrix to obtain at least one first matching error; wherein each point pair corresponds to a first matching error; For each of the point pairs, when a first matching error corresponding to the point pair is less than a third error threshold, determining a point in the point pair as a first inlier; When the number of the first interior points is greater than a first number, it is determined that the first homography matrix satisfies the first condition.

3. The method according to claim 1, characterized in that The method further comprises: Matching at least one matching point pair in the pair of first images based on the second homography matrix to obtain at least one second matching error; wherein each point pair corresponds to a second matching error; For each of the point pairs, when the second matching error corresponding to the point pair is less than a third error threshold, determining a point in the point pair as a second inner point; When the number of the second interior points is greater than the first number, it is determined that the second homography matrix satisfies the first condition.

4. The method according to claim 1, characterized in that: The method further comprises: When the first homography matrix satisfies the first condition and an absolute value of a difference between corresponding elements of the second homography matrix and the first homography matrix is ​​less than a preset value, it is determined that the second homography matrix satisfies the first condition.

5. The method according to any one of claims 1 to 4, characterized in that The duration used in the process of determining the first homography matrix is ​​a first duration, and the duration used in the process of determining the second homography matrix is ​​a second duration, and the method further includes: determining a second quantity based on a difference between the first duration and the second duration, the second quantity being positively correlated with the difference between the first duration and the second duration; Performing the first operation for the second number of times to obtain a set of target point pairs for determining the target homography matrix; The first operation includes: randomly determining a candidate point pair set from a plurality of matching point pairs in the pair of second images; wherein the target point pair set is determined from all candidate point pair sets.

6. The method according to any one of claims 1 to 4, characterized in that The method further comprises: Determining the second error threshold based on the error of the first homography matrix and the first error threshold; The difference between the second error threshold and the first error threshold is negatively correlated with the error of the first homography matrix.

7. The method according to any one of claims 1 to 4, characterized in that The method further comprises: When the second homography matrix does not satisfy the first condition, the first error threshold is output, and the first error threshold is used to determine the target homography matrix between the pair of second images, and the error of the target homography matrix is ​​within the first error threshold.

8. The method according to any one of claims 1 to 4, characterized in that The method further comprises: For at least one pair of second images, perform feature point matching on each pair of second images to obtain at least one symmetric matrix; wherein each of the symmetric matrices corresponds to a pair of second images; The Jacobi iteration method is used to update each of the symmetric matrices to obtain a target homography matrix between a pair of second images corresponding to each of the symmetric matrices.

9. The method according to claim 8, characterized in that The Jacobi iteration method is used to update each of the symmetric matrices to obtain a target homography matrix between a pair of second images corresponding to each of the symmetric matrices, including: Performing at least one second operation to obtain the eigenvalues ​​and eigenvectors of each of the symmetric matrices; wherein the second operation includes: determining the maximum element on the off-diagonal line of each of the symmetric matrices, and updating the elements of the row and column of each of the maximum elements in the corresponding symmetric matrix based on the rotation matrix corresponding to each of the maximum elements; Based on the eigenvalues ​​and eigenvectors of each of the symmetric matrices, a target homography matrix between a pair of second images corresponding to each of the symmetric matrices is determined.

10. The method according to claim 9, characterized in that The updating of the elements of the row and column of each of the maximum elements in the corresponding symmetric matrix based on the rotation matrix corresponding to each of the maximum elements comprises: Extracting the elements of the row and column where each of the maximum elements is located in the corresponding symmetric matrix to obtain an element set; Based on the rotation matrix corresponding to each of the maximum elements, updating the elements on the row and column where each of the maximum elements in the corresponding symmetric matrix is ​​located in the element set; Based on the position of each element before updating in the corresponding symmetric matrix, each element after updating is saved to a corresponding position in the corresponding symmetric matrix.

11. The method according to any one of claims 1 to 4, characterized in that The method further comprises: Based on the second error threshold, a target homography matrix between the pair of second images is determined.

12. An image processing device, characterized in that: The device comprises: An acquisition module, used for acquiring a pair of first images including the same object; A first determining module, configured to determine a first homography matrix between the pair of first images based on a first error threshold, wherein an error of the first homography matrix is ​​within the first error threshold; a second determining module, configured to determine, when the first homography matrix satisfies a first condition, a second homography matrix between the pair of first images based on a second error threshold, wherein an error of the second homography matrix is ​​within the second error threshold, wherein the second error threshold is greater than the first error threshold; A first output module is used to output the second error threshold when the second homography matrix satisfies the first condition, and the second error threshold is used to determine a target homography matrix between a pair of second images including the same object, and the error of the target homography matrix is ​​within the second error threshold; the target homography matrix is ​​used to process the pair of second images.

13. A communication device, characterized in that: The communication device comprises: A memory for storing computer executable instructions; A processor, connected to the memory, configured to implement the method according to any one of claims 1 to 11 by executing the computer executable instructions.

14. A chip, characterized in that: The chip comprises: A processor, configured to call and run a computer program from a memory, so that a device equipped with the chip executes a method as claimed in any one of claims 1 to 11.

15. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by at least one processor, the method according to any one of claims 1 to 11 is implemented.

Images