Method, device and equipment for detecting precision of camera rotation movement and storage medium

Abstract

The present disclosure provides a camera rotation motion precision detection method, device and equipment, and a storage medium, relates to the technical field of artificial intelligence, specifically to the technical field of computer vision, augmented reality, virtual reality, deep learning and the like, and can be applied to the scene of meta universe, AIGC and the like. The scheme comprises: acquiring a first image and a second image photographed by a camera in a rotation process; determining a plurality of first feature points in the first image and a plurality of second feature points in the second image; determining an optimal relative rotation matrix of the first image and the second image based on first position information of the first feature points in the first image and second position information of the second feature points in the second image; calculating a deviation value of the first image and the second image based on the first position information of the first feature points, the second position information of the second feature points and the optimal relative rotation matrix; and determining a rotation precision result of the camera based on a comparison result of the deviation value and a preset deviation threshold.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of artificial intelligence, in particular to the technical field of computer vision, augmented reality, virtual reality, deep learning, etc., and can be applied to the scene of meta universe, AIGC, etc. BACKGROUND

[0002] In some application scenarios of a camera, the camera needs to shoot multiple frames of images in a rotating process. In such a scenario, the rotating shaft of the camera usually needs to be relatively fixed and not to have a large translation or tilt motion, so as to ensure the accuracy of the rotating motion of the camera and thus ensure the image quality of the camera. However, there is a lack of a method for effectively detecting the accuracy of the rotating motion of the camera in the related art. SUMMARY

[0003] The present disclosure provides a method, apparatus, device and storage medium for detecting the accuracy of the rotating motion of a camera.

[0004] According to a first aspect of the present disclosure, a method for detecting the accuracy of the rotating motion of a camera is provided, and the method comprises:

[0005] obtaining a first image and a second image shot by the camera in a rotating process;

[0006] determining a plurality of first feature points in the first image and a plurality of second feature points in the second image, wherein the first feature points and the second feature points correspond to each other in one-to-one manner;

[0007] determining an optimal relative rotation matrix of the first image and the second image based on first position information of the first feature points in the first image and second position information of the second feature points in the second image;

[0008] calculating a deviation value of the first image and the second image based on the first position information of the first feature points, the second position information of the second feature points and the optimal relative rotation matrix;

[0009] determining a rotating accuracy result of the camera based on a comparison result of the deviation value and a preset deviation threshold.

[0010] In some embodiments of the present disclosure, the determination of the optimal relative rotation matrix of the first image and the second image based on the first position information of the first feature points in the first image and the second position information of the second feature points in the second image comprises:

[0011] determining an intrinsic matrix of the camera;

[0012] calculating a first vector of the first feature points in a camera coordinate system corresponding to the first image based on the intrinsic matrix and the first position information of the first feature points in the first image;

[0013] based on the first vector of the first feature point and the second vector of the second feature point, determining an optimal relative rotation matrix of the first image and the second image.

[0014] based on the first vector of the first feature point and the second vector of the second feature point, determining an optimal relative rotation matrix of the first image and the second image.

[0015] In some embodiments of the present disclosure, based on the first vector of the first feature point and the second vector of the second feature point, determining an optimal relative rotation matrix of the first image and the second image comprises:

[0016] forming a vector pair by the first vector of each first feature point and the second vector of the second feature point corresponding to the first feature point to obtain a plurality of vector pairs;

[0017] based on the plurality of vector pairs, calculating K candidate relative rotation matrices of the first image and the second image, wherein each candidate relative rotation matrix corresponds to a vector pair, and K is an integer greater than 1;

[0018] based on each candidate relative rotation matrix, calculating the re-projection error of the corresponding first feature point and second feature point;

[0019] based on the re-projection error corresponding to each candidate relative rotation matrix, determining an optimal relative rotation matrix of the first image and the second image from the K candidate relative rotation matrices.

[0020] In some embodiments of the present disclosure, K candidate relative rotation matrices are obtained by performing the following process K times:

[0021] selecting a candidate vector pair from the plurality of vector pairs;

[0022] based on the first vector and the second vector in the candidate vector pair, generating a correction matrix;

[0023] decomposing the correction matrix into a product of a first orthogonal matrix, a diagonal matrix, and a second orthogonal matrix;

[0024] based on the first orthogonal matrix, the diagonal matrix, and the second orthogonal matrix, calculating a candidate relative rotation matrix of the first image and the second image.

[0025] In some embodiments of the present disclosure, based on the first orthogonal matrix, the diagonal matrix, and the second orthogonal matrix, calculating a candidate relative rotation matrix of the first image and the second image comprises:

[0026] based on the first orthogonal matrix and the second orthogonal matrix, constructing a discriminant matrix;

[0027] The first image and the second image are calculated based on a first orthogonal matrix, a second orthogonal matrix and a coefficient matrix, wherein the element values in the coefficient matrix are determined based on the discriminant matrix.

[0028] In some embodiments of the present disclosure, the re-projection error of each corresponding first feature point and second feature point is calculated based on each candidate relative rotation matrix, including:

[0029] For each candidate relative rotation matrix, a feature point conversion function is constructed based on the candidate relative rotation matrix and the intrinsic parameter matrix, and K feature point conversion functions are obtained.

[0030] For each feature point conversion function, the following steps are performed:

[0031] The second position information and the second vector of each second feature point are substituted into the feature point conversion function to calculate the projection position information of the projection point of each second feature point in the first image.

[0032] For each group of corresponding first feature points and second feature points, the re-projection error of the group of corresponding first feature points and second feature points is calculated based on the first position information of the first feature points and the projection position information of the projection points corresponding to the second feature points.

[0033] In some embodiments of the present disclosure, the optimal relative rotation matrix of the first image and the second image is determined from the K candidate relative rotation matrices based on the re-projection error result corresponding to each candidate relative rotation matrix, including:

[0034] For each feature point conversion function, each re-projection error calculated by the feature point conversion function is compared with a preset projection error threshold, and in response to the re-projection error being less than the preset projection error threshold, the second feature point corresponding to the re-projection error is determined as an inlier corresponding to the candidate relative rotation matrix in the feature point conversion function.

[0035] After determining the inliers corresponding to each candidate relative rotation matrix, the number of inliers corresponding to each candidate relative rotation matrix is counted, and the candidate relative rotation matrix with the largest number of inliers is determined as the optimal relative rotation matrix of the first image and the second image.

[0036] In some embodiments of the present disclosure, the deviation value of the first image and the second image is calculated based on the first position information of the first feature points, the second position information of the second feature points and the optimal relative rotation matrix, including:

[0037] A vector conversion function is constructed based on the optimal relative rotation matrix.

[0038] The first vector of each first feature point is substituted into a vector conversion function to calculate a projection vector of each first feature point in a camera coordinate system corresponding to the second image;

[0039] For each group of corresponding first feature points and second feature points, a rotation error angle of the group of corresponding first feature points and second feature points is calculated based on the projection vector of the first feature point and the second vector of the second feature point.

[0040] In some embodiments of the present disclosure, the deviation value of the first image and the second image is calculated based on the first position information of the first feature point, the second position information of the second feature point, and the optimal relative rotation matrix, and further includes: for each group of corresponding first feature points and second feature points, a disparity of the group of corresponding first feature points and second feature points is calculated based on the first position information of the first feature point and the second position information of the second feature point.

[0041] In some embodiments of the present disclosure, the rotation accuracy result of the camera is determined based on a comparison result of the deviation value and a preset deviation threshold, including:

[0042] A target rotation error angle is determined based on the plurality of rotation error angles;

[0043] In response to the target rotation error angle being less than a preset rotation error threshold, it is determined that the rotation accuracy of the camera meets the accuracy standard;

[0044] In response to the target rotation error angle not being less than the preset rotation error threshold, it is determined that the rotation accuracy of the camera does not meet the accuracy standard.

[0045] In some embodiments of the present disclosure, the rotation accuracy result of the camera is determined based on a comparison result of the deviation value and a preset deviation threshold, including:

[0046] A target rotation error angle is determined based on the plurality of rotation error angles, and a target disparity is determined based on the plurality of disparities;

[0047] In response to the target rotation error angle being less than a preset rotation error threshold and the target disparity being greater than a preset disparity threshold, it is determined that the rotation accuracy of the camera meets the accuracy standard;

[0048] In response to the target rotation error angle not being less than the preset rotation error threshold and / or the target disparity not being greater than the preset disparity threshold, it is determined that the rotation accuracy of the camera does not meet the accuracy standard.

[0049] According to a second aspect of the present disclosure, a camera rotation motion accuracy detection device is provided, which includes an image acquisition module, a feature point determination module, a matrix determination module, a deviation calculation module, and an accuracy determination module.

[0050] The image acquisition module is configured to acquire a first image and a second image captured by the camera during the rotation;

[0051] The feature point determination module is configured to determine a plurality of first feature points in the first image and a plurality of second feature points in the second image, wherein the first feature points correspond to the second feature points one by one;

[0052] The matrix determination module is configured to determine an optimal relative rotation matrix of the first image and the second image based on first position information of the first feature points in the first image and second position information of the second feature points in the second image;

[0053] The deviation calculation module is configured to calculate a deviation value of the first image and the second image based on the first position information of the first feature points, the second position information of the second feature points, and the optimal relative rotation matrix;

[0054] The precision determination module is configured to determine a rotation precision result of the camera based on a comparison result of the deviation value and a preset deviation threshold.

[0055] In some embodiments of the present disclosure, when the matrix determination module is configured to determine the optimal relative rotation matrix of the first image and the second image based on the first position information of the first feature points in the first image and the second position information of the second feature points in the second image, the matrix determination module is specifically configured to:

[0056] determine an intrinsic matrix of the camera;

[0057] calculate a first vector of each first feature point in a camera coordinate system corresponding to the first image based on the intrinsic matrix and the first position information of the first feature points in the first image;

[0058] calculate a second vector of each second feature point in a camera coordinate system corresponding to the second image based on the intrinsic matrix and the second position information of the second feature points in the second image;

[0059] determine the optimal relative rotation matrix of the first image and the second image based on the first vector of each first feature point and the second vector of the second feature point corresponding to the first feature point.

[0060] In some embodiments of the present disclosure, when the matrix determination module is configured to determine the optimal relative rotation matrix of the first image and the second image based on the first vector of each first feature point and the second vector of the second feature point, the matrix determination module is specifically configured to:

[0061] compose the first vector of each first feature point and the second vector of the second feature point corresponding to the first feature point into a vector pair to obtain a plurality of vector pairs;

[0062] determine K candidate relative rotation matrices of the first image and the second image based on the plurality of vector pairs, wherein each candidate relative rotation matrix corresponds to a vector pair, and K is an integer greater than 1;

[0063] calculate a re-projection error of each corresponding first feature point and second feature point based on each candidate relative rotation matrix;

[0064] determine an optimal relative rotation matrix of the first image and the second image from the K candidate relative rotation matrices based on the re-projection error results corresponding to each candidate relative rotation matrix.

[0065] In some embodiments of the present disclosure, the matrix determination module obtains the K candidate relative rotation matrices by performing the following process K times:

[0066] select a candidate vector pair from the plurality of vector pairs;

[0067] generate a correction matrix based on the first vector and the second vector in the candidate vector pair;

[0068] decompose the correction matrix into a product of a first orthogonal matrix, a diagonal matrix, and a second orthogonal matrix;

[0069] calculate a candidate relative rotation matrix of the first image and the second image based on the first orthogonal matrix, the diagonal matrix, and the second orthogonal matrix.

[0070] In some embodiments of the present disclosure, when the matrix determination module is used to calculate a candidate relative rotation matrix of the first image and the second image based on the first orthogonal matrix, the diagonal matrix, and the second orthogonal matrix, the matrix determination module is specifically used to:

[0071] construct a discriminant matrix based on the first orthogonal matrix and the second orthogonal matrix;

[0072] calculate a candidate relative rotation matrix of the first image and the second image based on the first orthogonal matrix, the second orthogonal matrix, and a coefficient matrix, wherein the element values in the coefficient matrix are determined based on the discriminant matrix.

[0073] In some embodiments of the present disclosure, when the matrix determination module is used to calculate a re-projection error of each corresponding first feature point and second feature point based on each candidate relative rotation matrix, the matrix determination module is specifically used to:

[0074] construct a feature point conversion function based on the candidate relative rotation matrix and the intrinsic matrix for each candidate relative rotation matrix, to obtain K feature point conversion functions;

[0075] perform the following steps for each feature point conversion function:

[0076] The second position information and the second vector of each second feature point are substituted into the feature point conversion function to calculate projection position information of a projection point of each second feature point in the first image;

[0077] For each group of corresponding first feature points and second feature points, a re-projection error of the group of corresponding first feature points and second feature points is calculated based on the first position information of the first feature points and the projection position information of the projection points corresponding to the second feature points.

[0078] In some embodiments of the present disclosure, the matrix determination module is specifically configured to:

[0079] For each feature point conversion function, each re-projection error calculated by the feature point conversion function is compared with a preset projection error threshold, and in response to the re-projection error being less than the preset projection error threshold, a second feature point corresponding to the re-projection error is determined as an inlier corresponding to a candidate relative rotation matrix in the feature point conversion function.

[0080] After the inliers corresponding to each candidate relative rotation matrix are determined, the number of inliers corresponding to each candidate relative rotation matrix is counted, and a candidate relative rotation matrix corresponding to the largest number of inliers is determined as the optimal relative rotation matrix of the first image and the second image.

[0081] In some embodiments of the present disclosure, the deviation calculation module is specifically configured to:

[0082] The vector conversion function is constructed based on the optimal relative rotation matrix;

[0083] The first vector of each first feature point is substituted into the vector conversion function to calculate a projection vector of each first feature point in the camera coordinate system corresponding to the second image.

[0084] For each group of corresponding first feature points and second feature points, a rotation error angle of the group of corresponding first feature points and second feature points is calculated based on the projection vector of the first feature points and the second vector of the second feature points.

[0085] In some embodiments of the present disclosure, the deviation calculation module, when calculating the deviation value of the first image and the second image based on the first position information of the first feature point, the second position information of the second feature point, and the optimal relative rotation matrix, is further configured to: for each set of corresponding first feature points and second feature points, calculate the parallax of the set of corresponding first feature points and second feature points based on the first position information of the first feature points and the second position information of the second feature points.

[0086] In some embodiments of the present disclosure, the precision determination module, when determining the rotation precision result of the camera based on the comparison result of the deviation value and the preset deviation threshold, is specifically configured to:

[0087] determine a target rotation error angle based on the plurality of rotation error angles;

[0088] in response to the target rotation error angle being less than the preset rotation error threshold, determine that the rotation precision of the camera meets the precision standard;

[0089] in response to the target rotation error angle not being less than the preset rotation error threshold, determine that the rotation precision of the camera does not meet the precision standard.

[0090] In some embodiments of the present disclosure, the precision determination module, when determining the rotation precision result of the camera based on the comparison result of the deviation value and the preset deviation threshold, is specifically configured to:

[0091] determine a target rotation error angle based on the plurality of rotation error angles, and determine a target parallax based on the plurality of parallaxes;

[0092] in response to the target rotation error angle being less than the preset rotation error threshold and the target parallax being greater than the preset parallax threshold, determine that the rotation precision of the camera meets the precision standard;

[0093] in response to the target rotation error angle not being less than the preset rotation error threshold and / or the target parallax not being greater than the preset parallax threshold, determine that the rotation precision of the camera does not meet the precision standard.

[0094] According to a third aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method provided in the first aspect.

[0095] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, the computer instructions being used to enable a computer to perform the method provided in the first aspect.

[0096] According to a fifth aspect of the present disclosure, a computer program product is provided, comprising a computer program which, when executed by a processor, implements the method according to the first aspect.

[0097] It should be understood that the content described in this part is not intended to identify key or important features of the embodiments of the present disclosure, nor to limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description.

[0098] The technical scheme provided by the present disclosure has the beneficial effects that:

[0099] The precision detection method for camera rotation motion provided by the embodiments of the present disclosure can accurately calculate the deviation value of the two frames of images based on the data of the corresponding feature points of the two frames of images captured by the camera during the rotation process, and accurately and objectively determine the precision of the camera rotation motion based on the deviation value. BRIEF DESCRIPTION OF DRAWINGS

[0100] The accompanying drawings are used to better understand the present scheme and do not constitute a limitation on the present disclosure. Among them:

[0101] Figure 1 A schematic diagram of an image captured by a camera during a rotation process is shown;

[0102] Figure 2 A flowchart of a precision detection method for camera rotation motion is shown;

[0103] Figure 3 A flowchart of determining an optimal relative rotation matrix of a first image and a second image is shown;

[0104] Figure 4 A specific flowchart of implementing S340 is shown;

[0105] Figure 5 A schematic diagram of a precision detection device for camera rotation motion is shown;

[0106] Figure 6 A schematic block diagram of an example electronic device that can be used to implement the present disclosure is shown. DETAILED DESCRIPTION

[0107] Exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings, which include various details of the embodiments of the present disclosure to help understanding, and should be considered as merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Also, in order to be clear and concise, the description below omits the description of well-known functions and structures.

[0108] It should be understood that, in the embodiments of the present disclosure, the character " / " generally represents that the associated objects before and after are in an "or" relationship. The terms "first", "second", etc. are only for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features.

[0109] In some application scenarios of the camera, the camera needs to shoot multiple frames of images in the rotating process. Taking the shooting of a panoramic image as an example, the camera needs to rotate a circle to shoot multiple images at different angle positions, and then stitch the multiple images at different angle positions into a panoramic image. In the above type of scene, the rotating shaft of the camera usually needs to be relatively fixed and not to have a large translation or tilt motion, so as to ensure the accuracy of the rotating motion of the camera and thus ensure the image quality of the camera. However, there is a lack of a method for effectively detecting the accuracy of the rotating motion of the camera in the related art.

[0110] The execution subject of the method can be a terminal device, or a computer, or a server, or can also be other devices with data processing capability. The execution subject of the method is not limited herein. In some embodiments, the execution subject of the camera rotating motion accuracy detection method provided by the embodiments of the present disclosure can be a terminal device (such as a vehicle-mounted computer) on a host vehicle.

[0111] Optionally, the terminal device can be a mobile phone, or can also be a tablet computer, a wearable device, a vehicle-mounted device, an augmented reality (AR) / virtual reality (VR) device, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), etc. The specific type of the terminal device is not limited in the embodiments of the present disclosure.

[0112] In some embodiments, the server can be a single server, or can also be a server cluster composed of multiple servers. In some embodiments, the server cluster can also be a distributed cluster. The specific implementation of the server is not limited in the present disclosure.

[0113] The camera rotating motion accuracy detection method provided by the present disclosure is exemplarily described below.

[0114] Figure 1 A schematic diagram of the camera provided by the present disclosure for shooting images in the rotating process is shown. In the schematic diagram, the camera is rotating around the rotating shaft, and the camera is shooting images in the rotating process. Figure 1In some embodiments, the coordinate system OXYZ is a coordinate system of the camera. Taking the Z-axis as the rotation axis of the camera as an example, during the rotation of the camera, it is generally expected that the Z-axis can be relatively fixed and does not undergo large translation or tilting, and the like, and in this case, the rotation accuracy of the camera can be considered to be relatively high. If large translation or tilting, and the like, occurs, the rotation accuracy of the camera can be considered to be relatively low. The camera rotation accuracy detection method provided in the embodiments of the present disclosure can detect the rotation accuracy of the camera by using the relevant information of two images captured by the camera during the rotation.

[0115] Figure 2 FIG. 1 shows a flowchart of a camera rotation accuracy detection method provided in the present disclosure, as shown in FIG. 1, the method can mainly include the following steps: Figure 2

[0116] S210: Obtain a first image and a second image captured by the camera during the rotation.

[0117] It can be understood that in some scenarios, the camera needs to be rotated to capture an image, and the camera can capture images at different angle positions during the rotation. Taking the capture of a panoramic image as an example, the camera needs to be rotated for one revolution to capture multiple images at different angle positions, and then the multiple images at different angle positions are spliced into a panoramic image. Here, the first image and the second image are two images captured by the camera at different angle positions during the rotation, and the first image and the second image need to contain overlapping content.

[0118] S220: Determine a plurality of first feature points in the first image and a plurality of second feature points in the second image.

[0119] In S220, the first feature points and the second feature points are one-to-one corresponding, for example, N first feature points are determined in the first image, and N second feature points are also determined in the second image at the same time, where N is an integer greater than 1.

[0120] Optionally, a plurality of sampling points can be determined from the first image based on a preset feature recognition algorithm, and then a light flow tracking algorithm is used to predict the positions of the sampling points in the first image in the second image, and the sampling points commonly contained in the first image and the second image are defined as feature points, where the feature points are defined as first feature points in the first image, and the feature points are defined as second feature points in the second image.

[0121] ​It can be understood that after the plurality of first feature points are determined in the first image, the position information of each first feature point in the first image can be determined, and the position information of the first feature point in the first image is defined as first position information; after the plurality of second feature points are determined in the second image, the position information of each second feature point in the second image can be determined, and the position of the second feature point in the second image is defined as second position information.

[0122] Here, taking the number of the first feature points and the second feature points as N as an example, the first position information of the N first feature points can be denoted as The second position information of the N second feature points can be denoted as It can be understood that and are a group of corresponding first feature points and second feature points, and thus there are N groups of corresponding first feature points and second feature points.

[0123] S230: determining an optimal relative rotation matrix of the first image and the second image based on the first position information of the first feature points in the first image and the second position information of the second feature points in the second image.

[0124] In S230, the vector of the feature point can be determined based on the position information of the feature point, and then the optimal relative rotation matrix of the first image and the second image is determined based on the intrinsic matrix of the camera and the determined vector, and the specific process of determining the optimal relative rotation matrix will be further introduced in the subsequent content, which will not be described here.

[0125] S240: calculating a deviation value of the first image and the second image based on the first position information of the first feature points, the second position information of the second feature points, and the optimal relative rotation matrix.

[0126] The present disclosure determines the vector corresponding to each feature point based on the intrinsic matrix and the position information of the feature point, and since the vector only has length and direction, it is convenient to convert the feature points in the second image to the first image based on the vector, so as to more directly compare the differences of the corresponding feature points in the first image and the second image, thereby accurately determining the rotation accuracy of the camera.

[0127] Optionally, the rotation error angle of the corresponding first feature points and second feature points can be determined based on the first position information of the first feature points, the second position information of the second feature points, and the optimal relative rotation matrix, and the deviation value of the first image and the second image is determined based on the rotation error angle.

[0128] Further, based on the determination of the deviation value of the first image and the second image based on the rotation error angle, the disparity of the corresponding first feature point and the second feature point can also be determined based on the first position information of the first feature point and the second position information of the second feature point, and another deviation value of the first image and the second image can be determined based on the disparity.

[0129] S250: Determine the rotation accuracy result of the camera based on the comparison result of the deviation value and the preset deviation threshold.

[0130] Here, the rotation accuracy result is used to indicate whether the rotation accuracy of the camera meets the accuracy standard. Specifically, if the comparison result meets the preset condition, indicating that the rotation accuracy of the camera is high, it can be determined that the rotation accuracy of the camera meets the accuracy standard; if the comparison result does not meet the preset condition, indicating that the rotation accuracy of the camera is low, it can be determined that the rotation accuracy of the camera does not meet the accuracy standard.

[0131] The camera rotation motion accuracy detection method provided by the embodiments of the present disclosure can accurately calculate the deviation value of the two frames of images based on the data of the corresponding feature points of the two frames of images captured by the camera during the rotation process, and accurately and objectively determine the accuracy of the rotation motion of the camera based on the deviation value.

[0132] Figure 3 A flowchart for determining an optimal relative rotation matrix of a first image and a second image is shown, as shown in Figure 3 The flowchart mainly includes the following steps:

[0133] S310: Determine the intrinsic matrix of the camera.

[0134] Here, the elements in the intrinsic matrix of the camera are related to the focal length and optical center of the camera. Specifically, the intrinsic matrix of the camera can be represented as K, and the intrinsic matrix of the camera is as follows:

[0135]

[0136] In the intrinsic matrix K, where f x =f / d x , f y =f / d y , f is the focal length of the camera, f x and f y are the normalized focal lengths on the u-axis and v-axis of the image captured by the camera, d x and d y represent the size of a unit pixel on the u-axis and v-axis, respectively, and c x and c x represent the optical center coordinates, i.e., the intersection of the optical axis of the camera and the image plane captured by the camera.

[0137] S320: Calculate a first vector of the first feature point in a camera coordinate system corresponding to the first image based on the intrinsic matrix and first position information of the first feature point in the first image.

[0138] Here, taking the first feature point as an example, the specific process of calculating the first vector of the first feature point in the camera coordinate system corresponding to the first image is introduced as follows:

[0139] It should be noted that, is represented as a 3X1 matrix. Then, normalization can be performed on , that is, wherein, is the vector norm.

[0140] S330: Calculate a second vector of the second feature point in a camera coordinate system corresponding to the second image based on the intrinsic matrix and second position information of the second feature point in the second image.

[0141] Here, the calculation process of the first vector of the first feature point may be referred to for calculating the second vector of the second feature point in the camera coordinate system corresponding to the second image, and details are not repeated here.

[0142] It should be noted that the first vector and the second vector can both be unit vectors.

[0143] S340: Determine an optimal relative rotation matrix of the first image and the second image based on the first vector of the first feature point and the second vector of the second feature point.

[0144] In S340, a plurality of candidate relative rotation matrices can be calculated based on a plurality of groups of the first vector of the first feature point and the second vector of the second feature point corresponding to each other, and then the optimal relative rotation matrix is determined from the plurality of candidate relative rotation matrices.

[0145] Figure 4 A specific flowchart for implementing S340 is shown in the disclosure, as shown in FIG. 4, which mainly includes the following steps: Figure 4

[0146] S410: Form a vector pair by combining the first vector of each first feature point with the second vector of the second feature point corresponding to the first feature point to obtain a plurality of vector pairs.

[0147] ​It can be understood that in S410, the first vector of a set of mutually corresponding first feature points and the second vector of a set of second feature points are taken as a vector pair. Taking the number of first feature points and the number of second feature points as N for example, 10 vector pairs can be formed, wherein, and form a vector pair.

[0148] S420: Calculate K candidate relative rotation matrices of the first image and the second image based on the plurality of vector pairs.

[0149] Here, each candidate relative rotation matrix corresponds to a vector pair, and K is an integer greater than 1. That is, K vector pairs need to be selected from the plurality of vector pairs, and a candidate relative rotation matrix is calculated based on each selected vector pair, so as to obtain K candidate relative rotation matrices of the first image and the second image.

[0150] Optionally, K candidate relative rotation matrices can be obtained by performing the following process K times:

[0151] (a): Select a candidate vector pair from the plurality of vector pairs.

[0152] Here, the first feature point and the second feature point are selected as a candidate vector pair.

[0153] (b): Generate a correction matrix based on the first vector and the second vector in the candidate vector pair.

[0154] Here, the correction matrix can be defined as M, and the correction matrix

[0155] (c): Decompose the correction matrix into the product of a first orthogonal matrix, a diagonal matrix and a second orthogonal matrix.

[0156] Here, the correction matrix M can be singular value decomposed (SVD) to obtain a decomposition result: M=VΣU T , wherein V is the first orthogonal matrix, Σ is the first orthogonal matrix, and U T is the second orthogonal matrix.

[0157] (d): Calculate a candidate relative rotation matrix of the first image and the second image based on the first orthogonal matrix, the diagonal matrix and the second orthogonal matrix.

[0158] Optionally, in step (d), a discriminant matrix can be constructed based on the first orthogonal matrix and the second orthogonal matrix. Specifically, the discriminant matrix can be represented as R, and the discriminant matrix R=VU TThen, a candidate relative rotation matrix of the first image and the second image can be calculated based on the first orthogonal matrix, the second orthogonal matrix and the coefficient matrix. Specifically, the candidate relative rotation matrix can be expressed as R 21 21 = VEU T wherein E is the coefficient matrix, and the element values in the coefficient matrix E are determined based on the discriminant matrix.

[0159] wherein when the discriminant matrix R is positive definite, when the discriminant matrix R is not positive definite,

[0160] It should be further noted that in the process of calculating the relative rotation matrix, a cost function J can be designed to minimize the alignment angle of the vectors of the feature points.

[0161] It can be understood that a candidate relative rotation matrix of the first image and the second image can be calculated every time steps (a) to (d) are executed; and K candidate relative rotation matrices can be obtained after K steps (a) to (d) are executed.

[0162] S430: Calculate the re-projection error of each corresponding first feature point and second feature point based on each candidate relative rotation matrix.

[0163] Optionally, in S430, the projection position information of the projection point of each second feature point in the first image can be calculated, and then the re-projection error of the corresponding first feature point and second feature point can be calculated based on the first position information of the first feature point and the projection position information of the projection point corresponding to the second feature point.

[0164] Specifically, for each candidate relative rotation matrix, a feature point conversion function is constructed based on the candidate relative rotation matrix and the intrinsic matrix, and K feature point conversion functions are obtained. The feature point conversion function is as follows:

[0165]

[0166]

[0167] wherein, is the projection point of the second feature point in the first image, is an intermediate parameter, is the first element in , is the second element in , is the first element in , is​ The second element in for The third element in This is the intrinsic parameter matrix of the camera.

[0168] For each feature point transformation function, perform the following steps:

[0169] Substitute the second position information and the second vector of each second feature point into the feature point transformation function to calculate the projection position information of the projection point of each second feature point in the first image; for each corresponding first feature point and second feature point, calculate the reprojection error of the corresponding first feature point and second feature point based on the first position information of the first feature point and the projection position information of the projection point corresponding to the second feature point.

[0170] Reprojection error can accurately reflect the difference between corresponding feature points in the first and second images. The smaller the reprojection error, the more accurate the transformation relationship expressed by the candidate relative rotation matrix. Based on the reprojection error, the rationality of the candidate relative rotation matrix can be accurately evaluated.

[0171] Taking the example that the number of the first feature point and the second feature point are both N, the projection position information of the projection point of the second feature point in the first image can be obtained by using the transformation function of each feature point, and N reprojection errors can be calculated.

[0172] S440: Based on the reprojection error results corresponding to each candidate relative rotation matrix, determine the optimal relative rotation matrix between the first image and the second image from the K candidate relative rotation matrices.

[0173] Multiple candidate relative rotation matrices are determined by selecting a pair of vectors. Then, the most reasonable relative rotation matrix is ​​selected from the candidate relative rotation matrices as the final optimal relative rotation matrix. This can avoid the interference of unreasonable feature points on the relative rotation matrix.

[0174] Optionally, for each feature point transformation function, each reprojection error calculated by the feature point transformation function is compared with a preset projection error threshold. In response to a reprojection error being less than the preset projection error threshold, the second feature point corresponding to the reprojection error is determined as the interior point corresponding to the candidate relative rotation matrix in the feature point transformation function. Specifically, the first feature point... Second feature point The reprojection error is defined as e i The projection point of the second feature point in the first image is defined as... Reprojection error e i for:

[0175] After determining the inliers corresponding to each candidate relative rotation matrix, the number of inliers corresponding to each candidate relative rotation matrix is counted, and the candidate relative rotation matrix corresponding to the largest number of inliers is determined as the optimal relative rotation matrix of the first image and the second image.

[0176] The re-projection error can accurately reflect the difference between the corresponding feature points in the first image and the second image. The smaller the re-projection error is, the more accurate the conversion relationship expressed by the candidate relative rotation matrix is. Since the inliers are determined by the re-projection error and the projection error threshold, the more inliers there are, the more accurate the conversion relationship expressed by the candidate relative rotation matrix is. The number of inliers can accurately evaluate the rationality of the candidate relative rotation matrix.

[0177] Optionally, in the embodiments of the present disclosure, S240 can specifically include:

[0178] Based on the optimal relative rotation matrix, a vector conversion function is constructed, the first vector of each first feature point is substituted into the vector conversion function, and the projection vector of each first feature point in the camera coordinate system corresponding to the second image is calculated. Here, the projection vector of the first feature point in the camera coordinate system corresponding to the second image is defined as the projection vector where R 21 * is the optimal relative rotation matrix, is the first feature point.

[0179] For each set of corresponding first feature points and second feature points, based on the projection vector of the first feature point and the second vector of the second feature point, the rotation error angle of the set of corresponding first feature points and second feature points is calculated. Specifically, the included angle between the projection vector of the first feature point and the second vector of the second feature point can be calculated, and the included angle is taken as the rotation error angle of the first feature point and the second feature point. Here, the rotation error angle of a set of corresponding first feature points and second feature points is defined as θ i the rotation error angle (<> represents the inner product of two vectors).

[0180] Optionally, in the case where the deviation value of the first image and the second image includes the rotation error angle of the first feature point and the second feature point, S250 can specifically include:

[0181] Based on the plurality of rotation error angles, a target rotation error angle is determined. Optionally, all rotation error angles (θ1, θ2, …, θ i , …, θ N) from small to large, and the median of all the rotation error angles is taken as the target rotation error angle; or, the average of the plurality of rotation error angles can be calculated, and the average is taken as the target rotation error angle.

[0182] After the target rotation error angle is determined, it can be determined that the rotation accuracy of the camera meets the accuracy criterion in response to the target rotation error angle being less than the preset rotation error threshold; or, it can be determined that the rotation accuracy of the camera does not meet the accuracy criterion in response to the target rotation error angle not being less than the preset rotation error threshold. Here, when the target rotation error angle is less than the preset rotation error threshold, it can be considered that the relative motion of the first image and the second image can be well fitted by a single rotation matrix without additional translation, indicating that the rotation position of the camera during rotation is relatively fixed, and the rotation accuracy of the camera is relatively high.

[0183] Optionally, in the case where the deviation value of the first image and the second image includes the rotation error angle of the first feature point and the second feature point, the disparity of the corresponding first feature point and second feature point can also be calculated, and the rotation error angle and the disparity are taken together as the deviation value of the first image and the second image.

[0184] Specifically, S240 can further include: for each group of corresponding first feature points and second feature points, calculating the disparity of the group of corresponding first feature points and second feature points based on the first position information of the first feature points and the second position information of the second feature points.

[0185] Optionally, in the case where the deviation value of the first image and the second image includes the rotation error angle and the disparity of the first feature point and the second feature point, S250 can specifically include:

[0186] The target rotation error angle is determined based on the plurality of rotation error angles, and the target disparity is determined based on the plurality of disparities.

[0187] Optionally, all the rotation error angles (θ1, θ2, …, θ i ,…θ N ) calculated in S240 can be sorted from small to large, and the median of all the rotation error angles is taken as the target rotation error angle; or, the average of the plurality of rotation error angles can be calculated, and the average is taken as the target rotation error angle.

[0188] Optionally, the disparity of a group of corresponding first feature points and second feature points can be defined as d i , and the disparity After obtaining the plurality of disparities, all the disparities (d1, d2, …, d i ,…d N ) d1, d2, …, d i ,…d NSort from small to large, take the median of all parallaxes as the target parallax; or, calculate the average of multiple parallaxes, and take the average as the target parallax.

[0189] After the target rotation error angle and the target parallax are determined, it can be determined that the rotation accuracy of the camera meets the accuracy criterion in response to the target rotation error angle being less than the preset rotation error threshold and the target parallax being greater than the preset parallax threshold; or, it can be determined that the rotation accuracy of the camera does not meet the accuracy criterion in response to the target rotation error angle not being less than the preset rotation error threshold and / or the target parallax not being greater than the preset parallax threshold.

[0190] Here, when the target rotation error angle is less than the preset rotation error threshold, it can be considered that the relative motion of the first image and the second image can be well fitted by a single rotation matrix without additional translation, indicating that the rotation position of the camera during rotation is relatively fixed, and the rotation accuracy of the camera is relatively high; at the same time, when the target parallax is greater than the preset parallax threshold, it indicates that the displacement of the first image and the second image captured by the camera during rotation is large enough, and the rotation motion is sufficient rather than a static state, the selection of the first image and the second image is reasonable, and the determination criterion of the rotation accuracy of the camera has high reliability.

[0191] Based on the same principle as the camera rotation motion accuracy detection method described above, the embodiments of the present disclosure provide a camera rotation motion accuracy detection device, Figure 5 A schematic diagram of a camera rotation motion accuracy detection device provided by the present disclosure is shown, as Figure 5 As shown in the figure, the camera rotation motion accuracy detection device 500 includes an image acquisition module 510, a feature point determination module 520, a matrix determination module 530, a deviation calculation module 540, and an accuracy determination module 550.

[0192] The image acquisition module 510 is configured to acquire a first image and a second image captured by the camera during rotation;

[0193] The feature point determination module 520 is configured to determine a plurality of first feature points in the first image and a plurality of second feature points in the second image, wherein the first feature points and the second feature points correspond one-to-one;

[0194] The matrix determination module 530 is configured to determine an optimal relative rotation matrix of the first image and the second image based on first position information of the first feature points in the first image and second position information of the second feature points in the second image;

[0195] The deviation calculation module 540 is configured to calculate a deviation value of the first image and the second image based on the first position information of the first feature points, the second position information of the second feature points, and the optimal relative rotation matrix;

[0196] The precision determination module 550 is configured to determine the rotation precision result of the camera based on a comparison result of the deviation value and the preset deviation threshold.

[0197] The precision detection device for camera rotation motion provided by the embodiments of the present disclosure can accurately calculate the deviation value of the two frames of images based on the data of the corresponding feature points of the two frames of images captured by the camera during the rotation process, and accurately and objectively determine the precision of the rotation motion of the camera based on the deviation value.

[0198] In some embodiments of the present disclosure, the matrix determination module 530, when used to determine the optimal relative rotation matrix of the first image and the second image based on the first position information of the first feature point in the first image and the second position information of the second feature point in the second image, is specifically configured to:

[0199] determine the intrinsic matrix of the camera;

[0200] calculate a first vector of the first feature point in the camera coordinate system corresponding to the first image based on the intrinsic matrix and the first position information of the first feature point in the first image;

[0201] calculate a second vector of the second feature point in the camera coordinate system corresponding to the second image based on the intrinsic matrix and the second position information of the second feature point in the second image;

[0202] determine the optimal relative rotation matrix of the first image and the second image based on the first vector of the first feature point and the second vector of the second feature point.

[0203] In some embodiments of the present disclosure, the matrix determination module 530, when used to determine the optimal relative rotation matrix of the first image and the second image based on the first vector of the first feature point and the second vector of the second feature point, is specifically configured to:

[0204] compose a vector pair by the first vector of each first feature point and the second vector of the second feature point corresponding to the first feature point to obtain a plurality of vector pairs;

[0205] calculate K candidate relative rotation matrices of the first image and the second image based on the plurality of vector pairs, wherein each candidate relative rotation matrix corresponds to a vector pair, and K is an integer greater than 1;

[0206] calculate the re-projection error of each corresponding first feature point and second feature point based on each candidate relative rotation matrix;

[0207] determine the optimal relative rotation matrix of the first image and the second image from the K candidate relative rotation matrices based on the re-projection error result corresponding to each candidate relative rotation matrix.

[0208] In some embodiments of the present disclosure, the matrix determining module 530 obtains the K candidate relative rotation matrices by performing the following process K times:

[0209] selecting a candidate vector pair from the plurality of vector pairs;

[0210] generating a correction matrix based on the first vector and the second vector in the candidate vector pair;

[0211] decomposing the correction matrix into a product of a first orthogonal matrix, a diagonal matrix, and a second orthogonal matrix;

[0212] calculating a candidate relative rotation matrix of the first image and the second image based on the first orthogonal matrix, the diagonal matrix, and the second orthogonal matrix.

[0213] In some embodiments of the present disclosure, when the matrix determining module 530 is configured to calculate a candidate relative rotation matrix of the first image and the second image based on the first orthogonal matrix, the diagonal matrix, and the second orthogonal matrix, the matrix determining module 530 is specifically configured to:

[0214] constructing a discriminant matrix based on the first orthogonal matrix and the second orthogonal matrix;

[0215] calculating a candidate relative rotation matrix of the first image and the second image based on the first orthogonal matrix, the second orthogonal matrix, and a coefficient matrix, wherein the element values in the coefficient matrix are determined based on the discriminant matrix.

[0216] In some embodiments of the present disclosure, when the matrix determining module 530 is configured to calculate the re-projection error of each corresponding first feature point and second feature point based on each candidate relative rotation matrix, the matrix determining module 530 is specifically configured to:

[0217] constructing, for each candidate relative rotation matrix, a feature point conversion function based on the candidate relative rotation matrix and the intrinsic matrix, to obtain K feature point conversion functions;

[0218] performing the following steps for each feature point conversion function:

[0219] substituting the second position information of each second feature point and the second vector into the feature point conversion function to calculate the projection position information of the projection point of each second feature point in the first image;

[0220] calculating, for each group of corresponding first feature points and second feature points, the re-projection error of the group of corresponding first feature points and second feature points based on the first position information of the first feature points and the projection position information of the projection points corresponding to the second feature points.

[0221] In some embodiments of the present disclosure, the matrix determination module 530, when determining the optimal relative rotation matrix of the first image and the second image from the K candidate relative rotation matrices based on the reprojection error results corresponding to each candidate relative rotation matrix, is specifically configured to:

[0222] For each feature point conversion function, compare each reprojection error calculated by the feature point conversion function with a preset projection error threshold, and in response to the reprojection error being less than the preset projection error threshold, determine the second feature point corresponding to the reprojection error as an inlier corresponding to the candidate relative rotation matrix in the feature point conversion function.

[0223] After determining the inliers corresponding to each candidate relative rotation matrix, count the number of inliers corresponding to each candidate relative rotation matrix, and determine the candidate relative rotation matrix with the largest number of inliers as the optimal relative rotation matrix of the first image and the second image.

[0224] In some embodiments of the present disclosure, the deviation calculation module 540, when calculating the deviation value of the first image and the second image based on the first position information of the first feature points, the second position information of the second feature points, and the optimal relative rotation matrix, is specifically configured to:

[0225] Construct a vector conversion function based on the optimal relative rotation matrix;

[0226] Substitute the first vector of each first feature point into the vector conversion function to calculate the projection vector of each first feature point in the camera coordinate system corresponding to the second image;

[0227] For each set of corresponding first feature points and second feature points, calculate the rotation error angle of the set of corresponding first feature points and second feature points based on the projection vector of the first feature points and the second vector of the second feature points.

[0228] In some embodiments of the present disclosure, the deviation calculation module 540, when calculating the deviation value of the first image and the second image based on the first position information of the first feature points, the second position information of the second feature points, and the optimal relative rotation matrix, is further configured to, for each set of corresponding first feature points and second feature points, calculate the parallax of the set of corresponding first feature points and second feature points based on the first position information of the first feature points and the second position information of the second feature points.

[0229] In some embodiments of the present disclosure, the precision determination module 550, when determining the rotation precision result of the camera based on the comparison result of the deviation value and the preset deviation threshold, is specifically configured to:

[0230] Determine the target rotation error angle based on the plurality of rotation error angles;

[0231] In response to the target rotation error angle being less than the preset rotation error threshold, it is determined that the rotation accuracy of the camera meets the accuracy criterion.

[0232] In response to the target rotation error angle being not less than the preset rotation error threshold, it is determined that the rotation accuracy of the camera does not meet the accuracy criterion.

[0233] In some embodiments of the present disclosure, the accuracy determination module 550 is specifically configured to:

[0234] The target rotation error angle is determined based on the plurality of rotation error angles, and the target parallax is determined based on the plurality of parallaxes.

[0235] In response to the target rotation error angle being less than the preset rotation error threshold and the target parallax being greater than the preset parallax threshold, it is determined that the rotation accuracy of the camera meets the accuracy criterion.

[0236] In response to the target rotation error angle being not less than the preset rotation error threshold and / or the target parallax being not greater than the preset parallax threshold, it is determined that the rotation accuracy of the camera does not meet the accuracy criterion.

[0237] It can be understood that the above-mentioned modules of the camera rotation motion accuracy detection device in the embodiments of the present disclosure have the functions of corresponding steps of the above-mentioned camera rotation motion accuracy detection method. The functions can be realized by hardware, or realized by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above-mentioned functions. The above-mentioned modules can be software and / or hardware, and the above-mentioned modules can be realized individually or realized in an integrated manner. The function description of each module of the camera rotation motion accuracy detection device can be specifically referred to the corresponding description of the camera rotation motion accuracy detection method, which will not be described here.

[0238] In the technical solutions of the present disclosure, the acquisition, storage and application of user personal information comply with relevant laws and regulations and do not violate public order and good customs.

[0239] According to the embodiments of the present disclosure, the present disclosure further provides an electronic device, a readable storage medium, and a computer program product.

[0240] In the exemplary embodiments, the electronic device includes at least one processor, and a memory connected in communication with the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method as described in the above embodiments. The electronic device can be the above-mentioned computer or server.

[0241] In an exemplary embodiment, the readable storage medium may be a non-transitory computer-readable storage medium storing computer instructions for causing a computer to perform the method described in the above embodiments.

[0242] In an exemplary embodiment, the computer program item includes a computer program that, when executed by a processor, implements the method described in the above embodiments.

[0243] Figure 6 A schematic block diagram of an example electronic device 1100 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0244] like Figure 6 As shown, the electronic device 600 includes a computing unit 601, which can perform various appropriate actions and processes based on a computer program stored in a read-only memory (ROM) 602 or a computer program loaded from a storage unit 608 into a random access memory (RAM) 603. The RAM 603 may also store various programs and data required for the operation of the device 600. The computing unit 601, ROM 602, and RAM 603 are interconnected via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0245] Multiple components in electronic device 600 are connected to I / O interface 605, including: input unit 606, such as keyboard, mouse, etc.; output unit 607, such as various types of displays, speakers, etc.; storage unit 608, such as disk, optical disk, etc.; and communication unit 609, such as network card, modem, wireless transceiver, etc. Communication unit 609 allows electronic device 600 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0246] The computing unit 601 can be various general and / or special purpose processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The computing unit 601 performs various methods and processes described above, such as the precision detection method of camera rotational motion. For example, in some embodiments, the precision detection method of camera rotational motion can be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 608. In some embodiments, part or all of the computer program can be loaded and / or installed onto the electronic device 600 via the ROM 602 and / or the communication unit 609. When the computer program is loaded onto the RAM 603 and executed by the computing unit 601, one or more steps of the precision detection method of camera rotational motion described above can be performed. Alternatively, in other embodiments, the computing unit 601 can be configured to perform the precision detection method of camera rotational motion by any other appropriate means, such as by means of firmware.

[0247] Various implementations of the systems and techniques described above can be realized in digital electronic circuitry, integrated circuitry, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on a chip (SOC), a programmable logic device (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0248] Program code for carrying out methods of the present disclosure can be written in any combination of one or more programming languages. The program code can be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the program code, when executed by the processor or controller, produces a means for implementing the functions / acts specified in the flowcharts and / or block diagrams. The program code can be executed entirely on a machine, partially on a machine, partially on a machine as a stand-alone software package, partially on a machine and partially on a remote machine or entirely on a remote machine or server.

[0249] In the context of this disclosure, a machine-readable medium can be a tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include but is not limited to an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0250] To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0251] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0252] The computer system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0253] It should be understood that the various forms of flow shown above can be used to reorder, add, or remove steps. For example, the steps recited in the present disclosure can be performed in parallel, in series, or in a different order, without limitation herein, so long as the desired results of the technology disclosed in the present disclosure are achieved.

[0254] The specific implementation described above does not constitute a limitation on the protection scope of the present disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent replacements, and improvements made within the spirit and principles of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A method for detecting accuracy of camera rotation movement, the method comprising: acquiring a first image and a second image captured by a camera during a rotation process; determining a plurality of first feature points in the first image and a plurality of second feature points in the second image, wherein the first feature points correspond to the second feature points one by one; determining an optimal relative rotation matrix of the first image and the second image based on first position information of the first feature points in the first image and second position information of the second feature points in the second image; calculating a deviation value of the first image and the second image based on the first position information of the first feature points, the second position information of the second feature points and the optimal relative rotation matrix; determining a rotation accuracy result of the camera based on a comparison result of the deviation value and a preset deviation threshold value; wherein the determining of the optimal relative rotation matrix of the first image and the second image based on the first position information of the first feature points in the first image and the second position information of the second feature points in the second image comprises: calculating a first vector corresponding to the first feature points based on an intrinsic matrix of the camera and the first position information, and calculating a second vector corresponding to the second feature points based on the intrinsic matrix and the second position information; calculating a plurality of candidate relative rotation matrices of the first image and the second image based on a plurality of vector pairs composed of the first vector and the second vector; determining the optimal relative rotation matrix from the plurality of candidate relative rotation matrices.

2. The method of claim 1, wherein, The determining of the optimal relative rotation matrix of the first image and the second image based on the first position information of the first feature points in the first image and the second position information of the second feature points in the second image comprises: determining an intrinsic matrix of the camera; calculating a first vector of the first feature points in a camera coordinate system corresponding to the first image based on the intrinsic matrix and the first position information of the first feature points in the first image; calculating a second vector of the second feature points in a camera coordinate system corresponding to the second image based on the intrinsic matrix and the second position information of the second feature points in the second image; determining the optimal relative rotation matrix of the first image and the second image based on the first vector of the first feature points and the second vector of the second feature points.

3. The method of claim 2, wherein, The determining of the optimal relative rotation matrix of the first image and the second image based on the first vector of the first feature points and the second vector of the second feature points comprises: forming a vector pair by combining the first vector of each first feature point with the second vector of the second feature point corresponding to the first feature point to obtain a plurality of vector pairs; calculating K candidate relative rotation matrices of the first image and the second image based on the plurality of vector pairs, wherein each candidate relative rotation matrix corresponds to one vector pair, and K is an integer greater than 1. calculating, based on each of the candidate relative rotation matrix, a re-projection error of each corresponding first feature point and second feature point; determining, based on the re-projection error result corresponding to each of the candidate relative rotation matrix, the optimal relative rotation matrix of the first image and the second image from the K candidate relative rotation matrix.

4. The method of claim 3, wherein, K candidate relative rotation matrices are obtained by performing the following process K times: selecting a candidate vector pair from the plurality of vector pairs; generating a correction matrix based on the first vector and the second vector in the candidate vector pair; decomposing the correction matrix into a product of a first orthogonal matrix, a diagonal matrix and a second orthogonal matrix; calculating, based on the first orthogonal matrix, the diagonal matrix and the second orthogonal matrix, a candidate relative rotation matrix of the first image and the second image.

5. The method of claim 4, wherein, The method for calculating, based on the first orthogonal matrix, the diagonal matrix and the second orthogonal matrix, a candidate relative rotation matrix of the first image and the second image comprises: constructing a discriminant matrix based on the first orthogonal matrix and the second orthogonal matrix; calculating, based on the first orthogonal matrix, the second orthogonal matrix and a coefficient matrix, a candidate relative rotation matrix of the first image and the second image, wherein the element values in the coefficient matrix are determined based on the discriminant matrix.

6. The method of claim 3, wherein, The method for calculating, based on each of the candidate relative rotation matrix, a re-projection error of each corresponding first feature point and second feature point comprises: constructing, based on the candidate relative rotation matrix and the intrinsic matrix, a feature point conversion function for each of the candidate relative rotation matrix, to obtain K feature point conversion functions; for each of the feature point conversion functions, the following steps are performed: substituting the second position information and the second vector of each of the second feature points into the feature point conversion function to calculate the projection position information of the projection point of each of the second feature points in the first image; for each corresponding first feature point and second feature point, calculating the re-projection error of the corresponding first feature point and second feature point based on the first position information of the first feature point and the projection position information of the projection point corresponding to the second feature point.

7. The method of claim 6, wherein, The method for determining, based on the re-projection error result corresponding to each of the candidate relative rotation matrix, the optimal relative rotation matrix of the first image and the second image from the K candidate relative rotation matrix comprises: for each of the feature point conversion functions, comparing each of the re-projection errors calculated by the feature point conversion function with a preset projection error threshold, and in response to the re-projection error being less than the preset projection error threshold, determining the second feature point corresponding to the re-projection error as an inlier corresponding to the candidate relative rotation matrix in the feature point conversion function; After determining the inliers corresponding to each of the candidate relative rotation matrices, the number of inliers corresponding to each of the candidate relative rotation matrices is counted, and the candidate relative rotation matrix corresponding to the largest number of inliers is determined as the optimal relative rotation matrix of the first image and the second image.

8. The method of any one of claims 2-7, wherein, The calculation of the deviation value of the first image and the second image based on the first position information of the first feature points, the second position information of the second feature points, and the optimal relative rotation matrix comprises: constructing a vector conversion function based on the optimal relative rotation matrix; the first vector of each of the first feature points is substituted into the vector conversion function to calculate the projection vector of each of the first feature points in the camera coordinate system corresponding to the second image; for each group of corresponding first feature points and second feature points, the rotation error angle of the group of corresponding first feature points and second feature points is calculated based on the projection vector of the first feature point and the second vector of the second feature point.

9. The method of claim 8, wherein, The calculation of the deviation value of the first image and the second image based on the first position information of the first feature points, the second position information of the second feature points, and the optimal relative rotation matrix further comprises: for each group of corresponding first feature points and second feature points, the disparity of the group of corresponding first feature points and second feature points is calculated based on the first position information of the first feature point and the second position information of the second feature point.

10. The method of claim 8, wherein, The determination of the rotation accuracy result of the camera based on the comparison result of the deviation value and the preset deviation threshold value comprises: determining a target rotation error angle based on a plurality of rotation error angles; in response to the target rotation error angle being less than a preset rotation error threshold value, determining that the rotation accuracy of the camera meets the accuracy standard; in response to the target rotation error angle being not less than the preset rotation error threshold value, determining that the rotation accuracy of the camera does not meet the accuracy standard.

11. The method of claim 9, wherein, The determination of the rotation accuracy result of the camera based on the comparison result of the deviation value and the preset deviation threshold value comprises: determining a target rotation error angle based on a plurality of rotation error angles, and determining a target disparity based on a plurality of disparities; in response to the target rotation error angle being less than a preset rotation error threshold value and the target disparity being greater than a preset disparity threshold value, determining that the rotation accuracy of the camera meets the accuracy standard; in response to the target rotation error angle being not less than the preset rotation error threshold value and / or the target disparity being not greater than the preset disparity threshold value, determining that the rotation accuracy of the camera does not meet the accuracy standard.

12. A camera rotation motion accuracy detection device, the device comprising: an image acquisition module configured to acquire a first image and a second image taken by a camera during rotation; a feature point determination module configured to determine a plurality of first feature points in the first image and a plurality of second feature points in the second image, wherein the first feature points and the second feature points correspond one-to-one; determine an optimal relative rotation matrix of the first image and the second image based on first position information of the first feature points in the first image and second position information of the second feature points in the second image; calculate a deviation value of the first image and the second image based on the first position information of the first feature points, the second position information of the second feature points and the optimal relative rotation matrix; determine a rotation accuracy result of the camera based on a comparison result of the deviation value and a preset deviation threshold value; In a possible implementation, when the matrix determining module is used to determine the optimal relative rotation matrix of the first image and the second image based on the first position information of the first feature points in the first image and the second position information of the second feature points in the second image, the matrix determining module is specifically configured to: calculate a first vector corresponding to each of the first feature points based on an intrinsic matrix of the camera and the first position information, and calculate a second vector corresponding to each of the second feature points based on the intrinsic matrix and the second position information; calculate a plurality of candidate relative rotation matrices of the first image and the second image based on a plurality of vector pairs composed of the first vector and the second vector; determine the optimal relative rotation matrix from the plurality of candidate relative rotation matrices.

13. The apparatus of claim 12, wherein, In a possible implementation, when the matrix determining module is used to determine the optimal relative rotation matrix of the first image and the second image based on the first position information of the first feature points in the first image and the second position information of the second feature points in the second image, the matrix determining module is specifically configured to: determine an intrinsic matrix of the camera; calculate a first vector of each of the first feature points in a camera coordinate system corresponding to the first image based on the intrinsic matrix and the first position information of the first feature points in the first image; calculate a second vector of each of the second feature points in a camera coordinate system corresponding to the second image based on the intrinsic matrix and the second position information of the second feature points in the second image; determine the optimal relative rotation matrix of the first image and the second image based on the first vector of the first feature point and the second vector of the second feature point.

14. The apparatus of claim 13, wherein, In a possible implementation, when the matrix determining module is used to determine the optimal relative rotation matrix of the first image and the second image based on the first vector of the first feature point and the second vector of the second feature point, the matrix determining module is specifically configured to: compose a vector pair by the first vector of each of the first feature points and the second vector of the second feature point corresponding to the first feature point to obtain a plurality of vector pairs; calculate K candidate relative rotation matrices of the first image and the second image based on the plurality of vector pairs, wherein each of the candidate relative rotation matrices corresponds to one of the vector pairs, and K is an integer greater than 1; calculate a re-projection error of each of the first feature point and the second feature point corresponding to each of the candidate relative rotation matrices. determine the optimal relative rotation matrix of the first image and the second image from the K candidate relative rotation matrices based on the re-projection error results corresponding to each of the candidate relative rotation matrices. 15.The apparatus of claim 14, wherein the matrix determination module obtains the K candidate relative rotation matrices by performing the following procedure K times: select a candidate vector pair from the plurality of vector pairs; generate a correction matrix based on the first vector and the second vector in the candidate vector pair; decompose the correction matrix into a product of a first orthogonal matrix, a diagonal matrix, and a second orthogonal matrix; calculate a candidate relative rotation matrix of the first image and the second image based on the first orthogonal matrix, the diagonal matrix, and the second orthogonal matrix.

16. The apparatus of claim 15, wherein, In a specific implementation, when the matrix determination module is used to calculate a candidate relative rotation matrix of the first image and the second image based on the first orthogonal matrix, the diagonal matrix, and the second orthogonal matrix, the matrix determination module is specifically configured to: construct a discriminant matrix based on the first orthogonal matrix and the second orthogonal matrix; calculate a candidate relative rotation matrix of the first image and the second image based on the first orthogonal matrix, the second orthogonal matrix, and a coefficient matrix, wherein the element values in the coefficient matrix are determined based on the discriminant matrix.

17. The apparatus of claim 14, wherein, In a specific implementation, when the matrix determination module is used to calculate the re-projection error of each corresponding first feature point and second feature point based on each candidate relative rotation matrix, the matrix determination module is specifically configured to: construct a feature point conversion function based on the candidate relative rotation matrix and the intrinsic matrix for each candidate relative rotation matrix, to obtain K feature point conversion functions; for each feature point conversion function, perform the following steps: substitute the second position information and the second vector of each second feature point into the feature point conversion function to calculate the projection position information of the projection point of each second feature point in the first image; for each group of corresponding first feature points and second feature points, calculate the re-projection error of the group of corresponding first feature points and second feature points based on the first position information of the first feature point and the projection position information of the projection point corresponding to the second feature point.

18. An electronic device, comprising: at least one processor; and a memory connected with the at least one processor in communication; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method in any one of claims 1-11. 19.A non-transitory computer-readable storage medium storing computer instructions for causing a computer to perform the method in any one of claims 1-11. 20.A computer program product comprising a computer program which, when executed by a processor, implements the method in any one of claims 1-11.

Images