Abnormal positioning method, device and storage medium for camera calibration

Abstract

The present disclosure relates to a camera calibration abnormality positioning method and device, and a storage medium. The method comprises: obtaining an image of a target object collected by a first camera of a terminal, to obtain a first image; obtaining an image of the target object collected by a second camera of a reference terminal, to obtain a second image, wherein the second camera is a camera in the reference terminal corresponding to the first camera of the terminal; calculating a difference parameter between the first camera and the second camera based on a matched feature pair in the first image and the second image; and determining an influencing factor of camera calibration abnormality according to the difference parameter. The above technical solution provides data basis for the determination process of the influencing factor of camera calibration abnormality, thereby helping to accurately locate the influencing factor causing the camera calibration abnormality, and further helping to improve production efficiency.

Description

Technical Field

[0001] This disclosure relates to the field of camera calibration technology, and in particular to anomaly localization methods, apparatus and storage media for camera calibration. Background Technology

[0002] As a crucial component of many digital products, cameras have become deeply integrated into users' lives. With the continuous development of technology, the performance requirements for cameras are also increasing. Among these requirements, camera calibration is a vital step, as the calibration results and accuracy directly affect the accuracy of subsequent operations of the imaging system.

[0003] In relevant scenarios, the camera calibration process may encounter anomalies due to interference from various factors, necessitating the identification of these influencing factors. In related technologies, this is typically done manually, analyzing abnormal samples. However, this method relies heavily on the experience of the personnel, and the results are often subjective, containing elements of guesswork. Summary of the Invention

[0004] To overcome the problems existing in related technologies, this disclosure provides a method, apparatus and storage medium for abnormal location of camera calibration.

[0005] According to a first aspect of the present disclosure, a method for anomaly localization in camera calibration is provided, comprising:

[0006] The first image is obtained by acquiring the image of the target object captured by the terminal's first camera;

[0007] The image of the target object captured by the second camera of the reference terminal is obtained to obtain a second image, wherein the second camera is the camera in the reference terminal that corresponds to the first camera of the terminal;

[0008] Based on the feature pairs that match in the first image and the second image, calculate the difference parameters between the first camera and the second camera;

[0009] The factors influencing camera calibration anomalies are determined based on the aforementioned difference parameters.

[0010] Optionally, the feature pair is a feature point pair, each feature point pair including a first feature point located in the first image and a second feature point located in the second image, and the step of calculating the difference parameter between the first camera and the second camera based on the matching feature pairs in the first image and the second image includes:

[0011] Calculate the first distance value between each first feature point in the multiple feature point pairs; and

[0012] Calculate a second distance value between each of the second feature points in the plurality of feature point pairs, wherein the difference parameter includes the first distance value and the second distance value.

[0013] Optionally, determining the influencing factors of camera calibration anomalies based on the difference parameters includes:

[0014] When the first distance value is greater than the second distance value, the factors affecting the camera calibration abnormality are determined to include the first camera being too close to focus;

[0015] When the first distance value is less than the second distance value, the factors affecting the camera calibration abnormality are determined to include the first camera focusing at a greater distance.

[0016] Optionally, the first camera includes a first target camera and a second target camera, and the first image includes a first target image captured by the first target camera and a second target image captured by the second target camera; the second camera includes a first reference camera and a second reference camera, and the second image includes a first reference image captured by the first reference camera and a second reference image captured by the second reference camera; the step of calculating the difference parameter between the first camera and the second camera based on the matching feature pairs in the first image and the second image includes:

[0017] Based on the feature point pairs that match the first target image and the first reference image, calculate the first rotation parameter and the first translation parameter between the first target camera and the first reference camera;

[0018] Based on the feature point pairs that match the second target image and the second reference image, calculate the second rotation parameter and the second translation parameter between the second target camera and the second reference camera;

[0019] The difference parameters include the first rotation parameter, the first translation parameter, the second rotation parameter, and the second translation parameter.

[0020] Optionally, determining the influencing factors of camera calibration anomalies based on the difference parameters includes:

[0021] Calculate the first difference value between the first rotation parameter and the second rotation parameter;

[0022] When the first difference value is greater than the rotation difference threshold, the factors affecting the camera calibration anomaly are determined to include the existence of rotation between the first target camera and the second target camera;

[0023] Calculate the second difference value between the first translation parameter and the second translation parameter;

[0024] When the second difference value is greater than the translation difference threshold, the factors affecting the camera calibration anomaly are determined to include the existence of translation between the first target camera and the second target camera.

[0025] Optionally, the method further includes:

[0026] The second target image is transformed based on the first rotation parameter and the first translation parameter;

[0027] Based on the feature point pairs that match the transformed second target image and the second reference image, the third rotation parameter and the third translation parameter between the first camera and the second camera are calculated.

[0028] Optionally, the first camera includes a first target camera and a second target camera, and the first image includes a first target image captured by the first target camera and a second target image captured by the second target camera; the second camera includes a first reference camera and a second reference camera, and the second image includes a first reference image captured by the first reference camera and a second reference image captured by the second reference camera; the step of calculating the difference parameter between the first camera and the second camera based on the matching feature pairs in the first image and the second image includes:

[0029] Based on the feature point pairs that match the first target image and the first reference image, calculate the first rotation parameter and the first translation parameter between the first target camera and the first reference camera;

[0030] The second target image is transformed based on the first rotation parameter and the first translation parameter;

[0031] Based on the feature point pairs that match the transformed second target image and the second reference image, a third rotation parameter and a third translation parameter are calculated between the first camera and the second camera. The difference parameters include the third rotation parameter and the third translation parameter.

[0032] According to a second aspect of the present disclosure, an anomaly location device for camera calibration is provided, comprising:

[0033] The first acquisition module is configured to acquire an image of the target object captured by the first camera of the terminal, and obtain a first image;

[0034] The second acquisition module is configured to acquire an image of the target object captured by the second camera of the reference terminal, and obtain a second image, wherein the second camera is a camera in the reference terminal that corresponds to the first camera of the terminal.

[0035] The first calculation module is configured to calculate the difference parameters between the first camera and the second camera based on the feature pairs that match in the first image and the second image.

[0036] The determination module is configured to determine the influencing factors of camera calibration anomalies based on the difference parameters.

[0037] Optionally, the feature pair is a feature point pair, each feature point pair including a first feature point located in the first image and a second feature point located in the second image, and the first calculation module includes:

[0038] The first calculation submodule is configured to calculate a first distance value between each first feature point in the plurality of feature point pairs based on the plurality of feature point pairs;

[0039] The second calculation submodule is configured to calculate a second distance value between each of the second feature points in the plurality of feature point pairs, wherein the difference parameter includes the first distance value and the second distance value.

[0040] Optionally, the determining module includes:

[0041] The first determining submodule is configured to determine, when the first distance value is greater than the second distance value, that the factors affecting the camera calibration abnormality include the first camera being too close to focus;

[0042] The second determining submodule is configured to determine, when the first distance value is less than the second distance value, that the factors affecting the camera calibration abnormality include the first camera being too far from focus.

[0043] Optionally, the first camera includes a first target camera and a second target camera, and the first image includes a first target image captured by the first target camera and a second target image captured by the second target camera; the second camera includes a first reference camera and a second reference camera, and the second image includes a first reference image captured by the first reference camera and a second reference image captured by the second reference camera; the first calculation module includes:

[0044] The third calculation submodule is configured to calculate the first rotation parameter and the first translation parameter between the first target camera and the first reference camera based on the feature point pairs that match in the first target image and the first reference image.

[0045] The fourth calculation submodule is configured to calculate the second rotation parameter and the second translation parameter between the second target camera and the second reference camera based on the feature point pairs that match in the second target image and the second reference image.

[0046] The difference parameters include the first rotation parameter, the first translation parameter, the second rotation parameter, and the second translation parameter.

[0047] Optionally, the determining module includes:

[0048] The fifth calculation submodule is configured to calculate a first difference value between the first rotation parameter and the second rotation parameter;

[0049] The third determining submodule is configured to determine that when the first difference value is greater than the rotation difference threshold, the influencing factors of the camera calibration abnormality include the existence of rotation between the first target camera and the second target camera.

[0050] The sixth calculation submodule is configured to calculate a second difference value between the first translation parameter and the second translation parameter;

[0051] The fourth determination submodule is configured to determine, when the second difference value is greater than the translation difference threshold, that the influencing factors of the camera calibration anomaly include translation between the first target camera and the second target camera.

[0052] Optionally, the device further includes:

[0053] The first image transformation module is configured to transform the second target image based on the first rotation parameter and the first translation parameter;

[0054] The second calculation module is configured to calculate a third rotation parameter and a third translation parameter between the first camera and the second camera based on the feature point pairs that match the transformed second target image and the second reference image.

[0055] Optionally, the first camera includes a first target camera and a second target camera, and the first image includes a first target image captured by the first target camera and a second target image captured by the second target camera; the second camera includes a first reference camera and a second reference camera, and the second image includes a first reference image captured by the first reference camera and a second reference image captured by the second reference camera; the first calculation module includes:

[0056] The seventh calculation submodule is configured to calculate the first rotation parameter and the first translation parameter between the first target camera and the first reference camera based on the feature point pairs that match in the first target image and the first reference image.

[0057] The image transformation submodule is configured to transform the second target image based on the first rotation parameter and the first translation parameter;

[0058] The eighth calculation submodule is configured to calculate a third rotation parameter and a third translation parameter between the first camera and the second camera based on the feature point pairs that match in the transformed second target image and the second reference image. The difference parameters include the third rotation parameter and the third translation parameter.

[0059] According to a third aspect of the present disclosure, an anomaly location device for camera calibration is provided, comprising:

[0060] processor;

[0061] Memory used to store processor-executable instructions;

[0062] The processor is configured as follows:

[0063] The first image is obtained by acquiring the image of the target object captured by the terminal's first camera;

[0064] The image of the target object captured by the second camera of the reference terminal is obtained to obtain a second image, wherein the second camera is the camera in the reference terminal that corresponds to the first camera of the terminal;

[0065] Based on the feature pairs that match in the first image and the second image, calculate the difference parameters between the first camera and the second camera;

[0066] The factors influencing camera calibration anomalies are determined based on the aforementioned difference parameters.

[0067] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided, having stored thereon computer program instructions that, when executed by a processor, implement the steps of the abnormal positioning method for camera calibration provided in the first aspect of the present disclosure.

[0068] The technical solutions provided by the embodiments of this disclosure have at least the following beneficial effects:

[0069] Images of the target object are captured by the first camera of the terminal and the second camera of the reference terminal, respectively, resulting in a first image and a second image. Furthermore, feature extraction and feature matching can be performed on the first and second images to obtain matching feature pairs. Based on these feature pairs, difference parameters between the first and second cameras can be calculated, and the influencing factors of camera calibration anomalies can be determined according to these difference parameters. In other words, the above technical solution provides data support for determining the influencing factors of camera calibration anomalies, thereby helping to accurately locate the factors causing these anomalies and ultimately improving production efficiency.

[0070] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0071] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0072] Figure 1 This is a flowchart illustrating an anomaly localization method for camera calibration according to an exemplary embodiment.

[0073] Figure 2 This is a flowchart illustrating the calculation of a difference parameter according to an exemplary embodiment.

[0074] Figure 3 This is a flowchart illustrating an anomaly localization method for camera calibration according to an exemplary embodiment.

[0075] Figure 4 This is a flowchart illustrating an anomaly localization method for camera calibration according to an exemplary embodiment.

[0076] Figure 5 This is a block diagram illustrating an anomaly location device for camera calibration according to an exemplary embodiment.

[0077] Figure 6 This is a block diagram illustrating an apparatus according to an exemplary embodiment. Detailed Implementation

[0078] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0079] Figure 1 This is a flowchart illustrating an anomaly localization method for camera calibration according to an exemplary embodiment, such as... Figure 1 As shown, the method may include the following steps.

[0080] In step S11, the image of the target object captured by the first camera of the terminal is obtained to obtain the first image.

[0081] The first camera can be a camera in the terminal that needs to be calibrated. The target object can be a target image, which can be presented in the form of a square grid, a combination of a square grid and dots, depending on the application scenario. For example, if the first camera can be the main camera of the terminal and the target object can be a square grid, then step S11 can refer to acquiring an image of the square grid through the main camera of the terminal to obtain a first image.

[0082] In step S12, the image of the target object captured by the second camera of the reference terminal is obtained to obtain the second image. Here, the reference terminal can be a standard terminal of the same type as the terminal mentioned above, and can be selected through relevant screening methods in specific implementations. For example, a terminal whose dual-camera calibration result difference is less than a first difference threshold and whose external parameters (such as rotation parameters, translation parameters) of dual-camera calibration in a single test are less than a second difference threshold (e.g., external parameters are close to 0) can be used as the reference terminal.

[0083] The second camera can be a camera in the reference terminal that corresponds to the first camera of the terminal. Continuing with the above example, when the first camera is the main camera of the terminal, the second camera can be the main camera of the reference terminal. Similarly, when the first camera is the macro lens of the terminal, the second camera is the macro lens of the reference terminal.

[0084] In step S13, the difference parameters between the first camera and the second camera are calculated based on the matching feature pairs in the first image and the second image.

[0085] The feature pairs can be obtained based on the first image and the second image. For example, in some implementation scenarios, feature extraction can be performed on the first image and the second image using corner extraction algorithms, edge detection algorithms, etc. When the target object is a square grid, the extracted features can be the corner points of the square grid and the pixel coordinates of each corner point. When the target object also includes dots, the extracted features may also include the center point of the dots and the pixel coordinates of the center point.

[0086] For the extracted features, matching methods such as optical flow can be used to obtain matching feature pairs. For example, when the extracted features are feature points, the obtained feature pairs can be feature point pairs, each pair including a first feature point located in the first image and a second feature point located in the second image. The first feature point and the second feature point can correspond to the same point in the target object.

[0087] In this way, the difference parameters between the first camera and the second camera can be calculated based on the matching feature pairs in the first image and the second image. For example, in one possible implementation, the first camera includes a first target camera and a second target camera. Here, the first target camera and the second target camera can be a pair of cameras that need to be calibrated simultaneously; in specific implementations, they can be any two cameras included in the terminal. The first image includes a first target image captured by the first target camera and a second target image captured by the second target camera.

[0088] The second camera includes a first reference camera and a second reference camera. The first reference camera is a camera in the reference terminal corresponding to the first target camera of the terminal, and the second reference camera is a camera in the reference terminal corresponding to the second target camera of the terminal. The second image includes a first reference image captured by the first reference camera and a second reference image captured by the second reference camera.

[0089] In this case, refer to Figure 2 The flowchart shown illustrates a calculation process for a difference parameter. Based on matching feature pairs in the first and second images, the difference parameter between the first and second cameras is calculated (step S13), including:

[0090] Based on the matching feature point pairs in the first target image and the first reference image, a first rotation parameter and a first translation parameter are calculated between the first target camera and the first reference camera. The second target image is then transformed based on these first rotation and translation parameters. In this way, the transformed second target image incorporates the pose differences between the first target camera and the first reference camera. Therefore, the transformed second target image can intuitively reflect the overall differences between the terminal's camera and the reference terminal's camera, which helps in locating the influencing factors of camera calibration anomalies.

[0091] Furthermore, based on the feature point pairs that match the transformed second target image and the second reference image, a third rotation parameter and a third translation parameter can be calculated between the first camera and the second camera. The difference parameters include the third rotation parameter and the third translation parameter.

[0092] It should be understood that since the transformed second target image incorporates the pose differences between the first target camera and the first reference camera, the third rotation parameter and the third translation parameter calculated based on the feature point pairs that match in the transformed second target image and the second reference image can quantitatively describe the overall difference between the terminal's camera and the reference terminal's camera.

[0093] Of course, in some implementation scenarios, the calculation process of the difference parameters may also involve the intrinsic parameters of the first camera and the second camera (such as focal length, principal point, etc.), and this disclosure does not impose any restrictions on this.

[0094] Thus, in step S14, the influencing factors of camera calibration anomalies are determined based on the difference parameters.

[0095] Following the above example, in some implementation scenarios, corresponding numerical ranges can be set for the third rotation parameter and the third translation parameter based on application requirements. When the degree of rotation represented by the third rotation parameter exceeds the rotation degree range, it is determined that the influencing factors of camera calibration anomaly include camera rotation at the terminal; when the amount of translation represented by the third translation parameter exceeds the translation amount range, it is determined that the influencing factors of camera calibration anomaly include camera translation at the terminal.

[0096] In the above technical solution, images of the target object are acquired by the first camera of the terminal and the second camera of the reference terminal, respectively, resulting in a first image and a second image. Furthermore, by performing feature extraction and feature matching on the first and second images, matching feature pairs between the two images can be obtained. This allows for the calculation of difference parameters between the first and second cameras based on these feature pairs, and the identification of influencing factors for camera calibration anomalies based on these difference parameters. In other words, the above technical solution provides data support for determining the influencing factors of camera calibration anomalies, thereby helping to accurately locate these factors. For example, in the production process, the above technical solution can quickly and accurately locate the influencing factors causing camera calibration anomalies, enabling timely handling of these factors and thus improving production efficiency.

[0097] Figure 3 This is a flowchart illustrating an anomaly localization method for camera calibration according to an exemplary embodiment of this disclosure, with reference to... Figure 3 The method includes:

[0098] S31, acquire the image of the target object captured by the terminal's first camera, and obtain the first image.

[0099] S32, acquire the image of the target object captured by the second camera of the reference terminal, and obtain the second image.

[0100] S33, calculate a first distance value between each first feature point in the plurality of feature point pairs. Each feature point pair includes a first feature point located in the first image and a second feature point located in the second image.

[0101] S34, calculate the second distance value between each second feature point in multiple feature point pairs, the difference parameter includes the first distance value and the second distance value.

[0102] For example, taking two pairs of feature points as an example, feature point pair 1 may include feature point A1 located in the first image and feature point B1 located in the second image; feature point pair 2 may include feature point A2 located in the first image and feature point B2 located in the second image. Thus, the distance between feature point A1 and feature point A2 can be calculated to obtain the first distance value. The distance between feature point B1 and feature point B2 can be calculated to obtain the second distance value. Here, the distance between feature points can be presented in a manner such as Euclidean distance. Furthermore, considering that the distance between feature points may be small, to improve the accuracy of the distance calculation, the square root formula in the Euclidean distance calculation can be replaced with a square formula.

[0103] After calculating the first distance value and the second distance value, the influencing factors of camera calibration anomalies can be determined based on the first distance value and the second distance value.

[0104] S35, when the first distance value is greater than the second distance value, it is determined that the factors affecting the camera calibration abnormality include the first camera being too close to focus.

[0105] S36, when the first distance value is less than the second distance value, it is determined that the factors affecting the camera calibration abnormality include the first camera focusing at a greater distance.

[0106] It should be noted that when a camera is capturing a target object, the distance at which it is focused can be represented as the object appearing smaller or larger in the image. Therefore, by calculating the first distance value between the first feature points in the first image and comparing it with the second distance value between the second feature points in the second image, the difference in focus between the terminal and the reference terminal can be determined. Furthermore, the first camera can be any camera that the terminal needs to calibrate, such as the terminal's main camera, secondary camera, etc.

[0107] In other words, the above technical solution can determine whether there is interference from focusing factors in the camera calibration process by calculating the first distance value between the first feature points in the first image and comparing it with the second distance value between the second feature points in the second image. Furthermore, if a difference in focusing is determined, the relevant focusing control chip and control strategy can be investigated to resolve the problem of abnormal camera calibration.

[0108] Figure 4 This is a flowchart illustrating an anomaly localization method for camera calibration according to an exemplary embodiment of this disclosure, with reference to... Figure 4 The method includes:

[0109] S41, acquire the image of the target object captured by the terminal's first camera, and obtain a first image. The first camera includes a first target camera and a second target camera, and the first image includes a first target image captured by the first target camera and a second target image captured by the second target camera.

[0110] S42, acquire the image of the target object captured by the second camera of the reference terminal, and obtain a second image. The second camera includes a first reference camera and a second reference camera, and the second image includes a first reference image captured by the first reference camera and a second reference image captured by the second reference camera.

[0111] S43, based on the feature point pairs that match in the first target image and the first reference image, calculate the first rotation parameter and the first translation parameter between the first target camera and the first reference camera.

[0112] S44, based on the feature point pairs matching in the second target image and the second reference image, calculate the second rotation parameter and the second translation parameter between the second target camera and the second reference camera. The difference parameters include the first rotation parameter, the first translation parameter, the second rotation parameter, and the second translation parameter.

[0113] For example, pose estimation can be performed based on the coordinate values ​​of the feature point pairs that match in the first target image and the first reference image to obtain the first rotation parameter and the first translation parameter; pose estimation can also be performed based on the coordinate values ​​of the feature point pairs that match in the second target image and the second reference image to obtain the second rotation parameter and the second translation parameter. For details on how to calculate the translation and rotation parameters based on the coordinate values ​​of the feature point pairs, please refer to the descriptions in related technologies. For the sake of brevity, these details are not elaborated here.

[0114] S45, calculate the first difference value between the first rotation parameter and the second rotation parameter.

[0115] S46, when the first difference value is greater than the rotation difference threshold, it is determined that the factors affecting the camera calibration anomaly include rotation between the first target camera and the second target camera.

[0116] S47, calculate the second difference value between the first translation parameter and the second translation parameter.

[0117] S48, when the second difference value is greater than the translation difference threshold, it is determined that the influencing factors of the camera calibration anomaly include the existence of translation between the first target camera and the second target camera.

[0118] The above technical solution can determine whether there is interference from camera rotation factors in the camera calibration process by calculating a first difference value between the first rotation parameter and the second rotation parameter, and by comparing the first difference value with a rotation difference threshold. Furthermore, if rotation exists between the first target camera and the second target camera, it can be determined that the terminal camera may be improperly assembled, and the relevant camera assembly steps can be checked and verified.

[0119] Furthermore, the aforementioned technical solution can also calculate a second difference value and compare it with a translation difference threshold to determine whether there is interference from camera translation factors in the camera calibration process. Further, if translation exists between the first target camera and the second target camera, it can be determined that the terminal camera may be improperly assembled and / or the terminal's OIS (Optical Image Stabilization) component may be faulty. In this case, the relevant camera assembly process and the terminal's OIS can be checked and verified.

[0120] Continue Figure 4 For example, in one possible implementation, if it is determined that the camera of the terminal has translation and / or rotation, the translation and rotation of the camera can also be quantified. In this case, the method further includes:

[0121] The second target image is transformed based on the first rotation parameters and the first translation parameters. In this way, the transformed second target image incorporates the pose differences between the first target camera and the first reference camera. Therefore, the transformed second target image can intuitively reflect the overall differences between the terminal's camera and the reference terminal's camera, which helps to locate the influencing factors of camera calibration anomalies.

[0122] Furthermore, based on the feature point pairs that match the transformed second target image and the second reference image, a third rotation parameter and a third translation parameter can be calculated between the first camera and the second camera. The difference parameters include the third rotation parameter and the third translation parameter.

[0123] It should be understood that since the transformed second target image incorporates the pose differences between the first target camera and the first reference camera, the third rotation parameter and the third translation parameter calculated based on the feature point pairs that match in the transformed second target image and the second reference image can quantitatively describe the overall difference between the terminal's camera and the reference terminal's camera.

[0124] Based on the same inventive concept, this disclosure also provides an anomaly positioning device for camera calibration. Figure 5This is a block diagram of an abnormal location device for camera calibration shown in an exemplary embodiment of the present disclosure. The device 500 includes:

[0125] The first acquisition module 501 is configured to acquire an image of the target object captured by the first camera of the terminal, and obtain a first image;

[0126] The second acquisition module 502 is configured to acquire an image of the target object captured by the second camera of the reference terminal, and obtain a second image, wherein the second camera is a camera in the reference terminal that corresponds to the first camera of the terminal.

[0127] The first calculation module 503 is configured to calculate the difference parameters between the first camera and the second camera based on the feature pairs that match in the first image and the second image.

[0128] The determination module 504 is configured to determine the influencing factors of camera calibration anomalies based on the difference parameters.

[0129] Optionally, the feature pair is a feature point pair, each feature point pair including a first feature point located in the first image and a second feature point located in the second image, and the first calculation module 503 includes:

[0130] The first calculation submodule is configured to calculate a first distance value between each first feature point in the plurality of feature point pairs based on the plurality of feature point pairs;

[0131] The second calculation submodule is configured to calculate a second distance value between each of the second feature points in the plurality of feature point pairs, wherein the difference parameter includes the first distance value and the second distance value.

[0132] Optionally, the determining module 504 includes:

[0133] The first determining submodule is configured to determine, when the first distance value is greater than the second distance value, that the factors affecting the camera calibration abnormality include the first camera being too close to focus;

[0134] The second determining submodule is configured to determine, when the first distance value is less than the second distance value, that the factors affecting the camera calibration abnormality include the first camera being too far from focus.

[0135] Optionally, the first camera includes a first target camera and a second target camera, and the first image includes a first target image captured by the first target camera and a second target image captured by the second target camera; the second camera includes a first reference camera and a second reference camera, and the second image includes a first reference image captured by the first reference camera and a second reference image captured by the second reference camera; the first calculation module 503 includes:

[0136] The third calculation submodule is configured to calculate the first rotation parameter and the first translation parameter between the first target camera and the first reference camera based on the feature point pairs that match in the first target image and the first reference image.

[0137] The fourth calculation submodule is configured to calculate the second rotation parameter and the second translation parameter between the second target camera and the second reference camera based on the feature point pairs that match in the second target image and the second reference image.

[0138] The difference parameters include the first rotation parameter, the first translation parameter, the second rotation parameter, and the second translation parameter.

[0139] Optionally, the determining module 504 includes:

[0140] The fifth calculation submodule is configured to calculate a first difference value between the first rotation parameter and the second rotation parameter;

[0141] The third determining submodule is configured to determine that when the first difference value is greater than the rotation difference threshold, the influencing factors of the camera calibration abnormality include the existence of rotation between the first target camera and the second target camera.

[0142] The sixth calculation submodule is configured to calculate a second difference value between the first translation parameter and the second translation parameter;

[0143] The fourth determination submodule is configured to determine, when the second difference value is greater than the translation difference threshold, that the influencing factors of the camera calibration anomaly include translation between the first target camera and the second target camera.

[0144] Optionally, the device 500 further includes:

[0145] The first image transformation module is configured to transform the second target image based on the first rotation parameter and the first translation parameter;

[0146] The second calculation module is configured to calculate a third rotation parameter and a third translation parameter between the first camera and the second camera based on the feature point pairs that match the transformed second target image and the second reference image.

[0147] Optionally, the first camera includes a first target camera and a second target camera, and the first image includes a first target image captured by the first target camera and a second target image captured by the second target camera; the second camera includes a first reference camera and a second reference camera, and the second image includes a first reference image captured by the first reference camera and a second reference image captured by the second reference camera; the first calculation module 503 includes:

[0148] The seventh calculation submodule is configured to calculate the first rotation parameter and the first translation parameter between the first target camera and the first reference camera based on the feature point pairs that match in the first target image and the first reference image.

[0149] The image transformation submodule is configured to transform the second target image based on the first rotation parameter and the first translation parameter;

[0150] The eighth calculation submodule is configured to calculate a third rotation parameter and a third translation parameter between the first camera and the second camera based on the feature point pairs that match in the transformed second target image and the second reference image. The difference parameters include the third rotation parameter and the third translation parameter.

[0151] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0152] This disclosure also provides an anomaly location device for camera calibration, comprising:

[0153] processor;

[0154] Memory used to store processor-executable instructions;

[0155] The processor is configured to perform the steps of the camera calibration anomaly localization method provided in this disclosure.

[0156] This disclosure also provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the steps of the abnormal positioning method for camera calibration provided in this disclosure.

[0157] Figure 6This is a block diagram illustrating an anomalous localization device 600 for camera calibration according to an exemplary embodiment. For example, device 600 may be a mobile phone, computer, messaging device, console, tablet device, personal digital assistant, etc.

[0158] Reference Figure 6 The device 600 may include one or more of the following components: a processing component 602, a memory 604, a power component 606, a multimedia component 608, an audio component 610, an input / output (I / O) interface 612, a sensor component 614, and a communication component 616.

[0159] Processing component 602 typically controls the overall operation of device 600, including operations associated with display, data communication, camera operation, and recording. Processing component 602 may include one or more processors 620 to execute instructions to complete all or part of the steps of the above-described camera calibration anomaly localization method. Furthermore, processing component 602 may include one or more modules to facilitate interaction between processing component 602 and other components. For example, processing component 602 may include a multimedia module to facilitate interaction between multimedia component 608 and processing component 602.

[0160] Memory 604 is configured to store various types of data to support the operation of device 600. Examples of this data include instructions, messages, pictures, videos, etc., for any application or method operating on device 600. Memory 604 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0161] The power supply component 606 provides power to the various components of the device 600. The power supply component 606 may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power to the device 600.

[0162] Multimedia component 608 includes a screen that provides an output interface between the device 600 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 608 includes a front-facing camera and / or a rear-facing camera. When the device 600 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0163] Audio component 610 is configured to output and / or input audio signals. For example, audio component 610 includes a microphone (MIC) configured to receive external audio signals when device 600 is in an operating mode, such as recording mode and voice recognition mode. The received audio signals may be further stored in memory 604 or transmitted via communication component 616. In some embodiments, audio component 610 also includes a speaker for outputting audio signals.

[0164] I / O interface 612 provides an interface between processing component 602 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0165] Sensor assembly 614 includes one or more sensors for providing status assessments of various aspects of device 600. For example, sensor assembly 614 can detect the on / off state of device 600, the relative positioning of components such as the display and keypad of device 600, changes in position of device 600 or any component of device 600, the presence or absence of user contact with device 600, orientation or acceleration / deceleration of device 600, and temperature changes of device 600. Sensor assembly 614 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 614 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 614 may also include an accelerometer, gyroscope, magnetometer, pressure sensor, or temperature sensor.

[0166] Communication component 616 is configured to facilitate wired or wireless communication between device 600 and other devices. Device 600 can access wireless networks based on communication standards, such as WiFi, 4G, or 5G, or combinations thereof. In one exemplary embodiment, communication component 616 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 616 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0167] In an exemplary embodiment, the device 600 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above-described camera calibration anomaly localization method.

[0168] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 604 including instructions, which can be executed by a processor 620 of the device 600 to complete the above-described abnormal location method for camera calibration. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0169] In another exemplary embodiment, a computer program product is also provided, the computer program product comprising a computer program executable by a programmable device, the computer program having a code portion for performing the above-described abnormal location method for camera calibration when executed by the programmable device.

[0170] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of this disclosure. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0171] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A method for anomaly localization using camera calibration, characterized in that, include: The first image is obtained by acquiring the image of the target object captured by the terminal's first camera; The image of the target object captured by the second camera of the reference terminal is obtained to obtain the second image. The second camera is the camera in the reference terminal that corresponds to the first camera of the terminal. The reference terminal includes a standard terminal of the same type as the terminal. The reference terminal is a terminal whose dual-camera calibration result difference is less than a first difference threshold and whose external parameters of dual-camera calibration are less than a second difference threshold in a single test. Based on the feature pairs that match in the first image and the second image, calculate the difference parameters between the first camera and the second camera; The factors influencing camera calibration anomalies are determined based on the aforementioned difference parameters.

2. The method according to claim 1, characterized in that, The feature pair is a feature point pair, and each feature point pair includes a first feature point located in the first image and a second feature point located in the second image. The step of calculating the difference parameter between the first camera and the second camera based on the matching feature pairs in the first image and the second image includes: Calculate the first distance value between each first feature point in the multiple feature point pairs; and Calculate a second distance value between each of the second feature points in the plurality of feature point pairs, wherein the difference parameter includes the first distance value and the second distance value.

3. The method according to claim 2, characterized in that, The step of determining the influencing factors of camera calibration anomalies based on the difference parameters includes: When the first distance value is greater than the second distance value, the factors affecting the camera calibration abnormality are determined to include the first camera being too close to focus; When the first distance value is less than the second distance value, the factors affecting the camera calibration abnormality are determined to include the first camera focusing at a greater distance.

4. The method according to claim 1, characterized in that, The first camera includes a first target camera and a second target camera, and the first image includes a first target image captured by the first target camera and a second target image captured by the second target camera; the second camera includes a first reference camera and a second reference camera, and the second image includes a first reference image captured by the first reference camera and a second reference image captured by the second reference camera; the step of calculating the difference parameter between the first camera and the second camera based on the matching feature pairs in the first image and the second image includes: Based on the feature point pairs that match the first target image and the first reference image, calculate the first rotation parameter and the first translation parameter between the first target camera and the first reference camera; Based on the feature point pairs that match the second target image and the second reference image, calculate the second rotation parameter and the second translation parameter between the second target camera and the second reference camera; The difference parameters include the first rotation parameter, the first translation parameter, the second rotation parameter, and the second translation parameter.

5. The method according to claim 4, characterized in that, The step of determining the influencing factors of camera calibration anomalies based on the difference parameters includes: Calculate the first difference value between the first rotation parameter and the second rotation parameter; When the first difference value is greater than the rotation difference threshold, the factors affecting the camera calibration anomaly are determined to include the existence of rotation between the first target camera and the second target camera; Calculate the second difference value between the first translation parameter and the second translation parameter; When the second difference value is greater than the translation difference threshold, the factors affecting the camera calibration anomaly are determined to include the existence of translation between the first target camera and the second target camera.

6. The method according to claim 5, characterized in that, The method further includes: The second target image is transformed based on the first rotation parameter and the first translation parameter; Based on the feature point pairs that match the transformed second target image and the second reference image, the third rotation parameter and the third translation parameter between the first camera and the second camera are calculated.

7. The method according to claim 1, characterized in that, The first camera includes a first target camera and a second target camera, and the first image includes a first target image captured by the first target camera and a second target image captured by the second target camera; the second camera includes a first reference camera and a second reference camera, and the second image includes a first reference image captured by the first reference camera and a second reference image captured by the second reference camera; the step of calculating the difference parameter between the first camera and the second camera based on the matching feature pairs in the first image and the second image includes: Based on the feature point pairs that match the first target image and the first reference image, calculate the first rotation parameter and the first translation parameter between the first target camera and the first reference camera; The second target image is transformed based on the first rotation parameter and the first translation parameter; Based on the feature point pairs that match the transformed second target image and the second reference image, a third rotation parameter and a third translation parameter are calculated between the first camera and the second camera. The difference parameters include the third rotation parameter and the third translation parameter.

8. An anomaly location device for camera calibration, characterized in that, include: The first acquisition module is configured to acquire an image of the target object captured by the first camera of the terminal, and obtain a first image; The second acquisition module is configured to acquire an image of the target object captured by the second camera of the reference terminal, and obtain a second image. The second camera is a camera in the reference terminal that corresponds to the first camera of the terminal. The reference terminal includes a standard terminal of the same type as the terminal. The reference terminal is a terminal whose dual-camera calibration result difference is less than a first difference threshold and whose external parameters of dual-camera calibration are less than a second difference threshold during a single test. The first calculation module is configured to calculate the difference parameters between the first camera and the second camera based on the feature pairs that match in the first image and the second image. The determination module is configured to determine the influencing factors of camera calibration anomalies based on the difference parameters.

9. An anomaly location device for camera calibration, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured as follows: The first image is obtained by acquiring the image of the target object captured by the terminal's first camera; The image of the target object captured by the second camera of the reference terminal is obtained to obtain the second image. The second camera is the camera in the reference terminal that corresponds to the first camera of the terminal. The reference terminal includes a standard terminal of the same type as the terminal. The reference terminal is a terminal whose dual-camera calibration result difference is less than a first difference threshold and whose external parameters of dual-camera calibration are less than a second difference threshold in a single test. Based on the feature pairs that match in the first image and the second image, calculate the difference parameters between the first camera and the second camera; The factors influencing camera calibration anomalies are determined based on the aforementioned difference parameters.

10. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When executed by a processor, the program instructions implement the steps of the method described in any one of claims 1 to 7.

Images