Camera parameter correction method and electronic equipment
By acquiring and processing the lens motion data when the camera is turned on the optical anti-shake module, the camera parameters are corrected, and the depth calculation errors caused by changes in the camera's internal and external parameters are solved, improving the image blur effect and user experience.
Patent Information
- Application Number
- CN202310548564.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-15
- Publication Date
- 2024-11-19
AI Technical Summary
When the camera turns on the optical anti-shake module, the internal and external parameters will change, resulting in the inability to correctly calculate the depth of each position in the image, affecting the image blur effect and user experience.
By obtaining the lens motion data when the camera turns on the optical anti-shake module, parameter corrections are performed, including data processing and corrections, to ensure that the camera can correctly calculate the depth of each position in the image.
It realizes accurate correction of internal and external parameters when the camera turns on the optical anti-shake module, ensuring the improvement of image blur effect and the improvement of user experience.
Smart Images

Figure CN118982583A_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present application relate to the field of data processing, and in particular to a camera parameter calibration method and electronic device. Background Art
[0002] The camera's intrinsic parameters and extrinsic parameters are two important parameters of the camera. Among them, the intrinsic parameters refer to the parameters related to the camera's own characteristics, such as focal length, pixel size, the position relationship between the center of the camera lens (i.e., the optical center) and the photosensitive element, etc. The extrinsic parameters refer to the parameters of the camera in the world coordinate system, such as the position of the optical center in the world coordinate system, the posture, the relative position relationship between the optical centers of the two cameras, etc.
[0003] The intrinsic and extrinsic parameters of a camera have many important uses. For example, in shooting scenes such as portraits where the subject needs to be highlighted, the camera often needs to blur the background in the image. In this case, the depth of each position in the image can be calculated using the intrinsic and extrinsic parameters of the camera, thereby distinguishing the subject from the background in the image.
[0004] However, when the camera turns on auxiliary shooting modules such as optical image stabilization, the internal and external parameters will change, causing the camera to be unable to correctly calculate the depth of each position in the image. Summary of the invention
[0005] The embodiments of the present application provide a camera parameter calibration method and an electronic device, which can calibrate the camera's internal and external parameters when the camera's optical image stabilization module is turned on, so as to ensure that the camera can correctly calculate the depth of each position in the image, thereby ensuring that the image has a good blurring effect and improving the user experience.
[0006] In order to achieve the above-mentioned purpose, the embodiment of the present application adopts the following technical solution.
[0007] In a first aspect, a method for calibrating camera parameters is provided, which is applied to an electronic device, wherein the electronic device includes a first camera equipped with an optical image stabilization module. The first image is any image captured by the first camera when the optical image stabilization module is turned on. The method includes: obtaining a first time period and a first parameter of the first camera. The first time period is an exposure time period for the first camera to capture the first image. The first parameter includes an internal parameter and an external parameter when the optical image stabilization module is not turned on. At least one set of lens motion data of the optical image stabilization module in the first time period is obtained. The lens motion data is used to indicate the offset of the lens of the optical image stabilization module relative to the optical center of the first camera. The first parameter is calibrated according to at least one set of lens motion data.
[0008] Based on this solution, the lens of the optical image stabilization module compensates for the shake of the first camera by displacement during the first time period. By acquiring the lens motion data during the first time period, the offset of the coordinate system of the first camera during the first time period can be obtained. Then, the first parameter can be corrected according to the coordinate transformation to obtain the internal and external parameters of the first camera when the optical image stabilization module is turned on. In this way, it is beneficial for the camera to correctly calculate the depth of each position in the image, thereby ensuring that the image has a better blur effect and improving the user experience.
[0009] In a possible design, the first parameter is corrected according to at least one set of lens motion data, including: performing a first process on at least one set of lens motion data to obtain at least one set of first data. The first process includes at least one of the following: filtering, data smoothing. The first parameter is corrected according to at least one set of first data. Based on this solution, the interference data in the data sample (at least one set of lens motion data) can be reduced, the accuracy of the data sample can be improved, and the correction of the first parameter can be more accurate.
[0010] In a possible design, the first parameter is corrected according to at least one set of first data, including: performing a second process on at least one set of first data to obtain a set of second data. The second process is any one of the following: calculating the mean, calculating the median, and calculating the mode. The first parameter is corrected according to the second data. Based on this scheme, the second data that best represents at least one set of first data is selected for subsequent calculation, which can greatly reduce the amount of data processing and improve the calculation speed while ensuring the accuracy of the calculation as much as possible.
[0011] In a possible design, the first parameter is corrected according to the second data, including: calculating the parameter change of the first camera according to the second data. The first parameter is corrected according to the parameter change of the first camera. Based on this solution, the second data that best represents at least one set of first data is selected to correct the first parameter, which can greatly reduce the amount of data processing and improve the correction speed while ensuring the accuracy of the correction as much as possible.
[0012] In a possible design, the amount of lens motion data in the first time period is the quotient of the first time period and the first sampling frequency. The first sampling frequency is the sampling frequency of the lens motion data by the drive of the optical image stabilization module. Based on this solution, the lens motion data can be made more representative and more representative of the movement of the lens in the first time period.
[0013] In a possible design, obtaining at least one set of lens motion data of the optical image stabilization module within a first time period includes: sending a first request to a first driver. The first driver is a driver of the optical image stabilization module. The first request is used to request the first driver to return at least one set of lens motion data of the optical image stabilization module within the first time period. Receive at least one set of lens motion data returned by the first driver. Based on this solution, the lens motion data can be obtained conveniently and accurately.
[0014] In a second aspect, a method for calculating image depth is provided, which is applied to an electronic device provided with at least two cameras. Each camera is equipped with an optical image stabilization module. In a second time period, each camera captures an image. The method includes: determining whether the corresponding optical image stabilization module of each camera is turned on in the second time period. If so, the first parameter of the corresponding camera is corrected by the camera parameter correction method of any one of the first aspects to obtain the second parameter of the corresponding camera. Among them, the first time period is the second time period. The depth of each position in each image is calculated according to the first parameter and the second parameter of each camera.
[0015] Based on this solution, when the optical image stabilization module is turned on when the binocular camera captures an image, the parameters of the camera with the optical image stabilization module turned on are first calibrated, and the depth of each position in the image is calculated according to the calibrated parameters, which is conducive to improving the accuracy of the calculation, ensuring the subsequent image blurring effect, and improving the user experience.
[0016] In one possible design, the depth of each position in each image is calculated based on the first parameters and the second parameters of each camera, including: calculating the depth of each position in each image based on the first parameters of the camera whose optical image stabilization module is not turned on in the second time period, and the second parameters of the camera whose optical image stabilization module is turned on in the second time period.
[0017] In a third aspect, an electronic device is provided, the electronic device comprising a camera equipped with an optical image stabilization module, one or more processors and one or more memories. The camera is connected to the one or more processors. The one or more memories are coupled to the one or more processors, and the one or more memories store computer instructions. When the one or more processors execute the computer instructions, the electronic device executes the camera parameter correction method of any one of the first aspects or the image depth calculation method of any one of the second aspects.
[0018] In a fourth aspect, a chip system is provided, wherein the chip includes a processing circuit and an interface. The processing circuit is used to call and run a computer program stored in a storage medium from a storage medium to execute the camera parameter correction method of any one of the first aspect or the image depth calculation method of any one of the second aspect.
[0019] In a fifth aspect, a computer-readable storage medium is provided, the computer-readable storage medium comprising computer instructions, and when the computer instructions are executed, the camera parameter correction method of any one of the first aspect or the image depth calculation method of any one of the second aspect is executed.
[0020] In a sixth aspect, a computer program product is provided, which includes instructions. When the computer program product runs on a computer, the computer can execute the camera parameter correction method of any one of the first aspects or the image depth calculation method of any one of the second aspects according to the instructions.
[0021] It should be understood that the technical solutions provided in the third, fourth, fifth and sixth aspects above and their technical features can all correspond to the camera parameter correction method provided in the first aspect and its possible design, or the image depth calculation method provided in the second aspect and its possible design, and therefore the beneficial effects that can be achieved are similar and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS
[0022] Figure 1 A schematic diagram of an imaging process of an object in a camera provided in an embodiment of the present application;
[0023] Figure 2 A schematic diagram of a virtual photosensitive element provided in an embodiment of the present application;
[0024] Figure 3 A schematic diagram of a process for calculating the depth of each position in an image provided in an embodiment of the present application;
[0025] Figure 4 A schematic diagram of radial distortion provided in an embodiment of the present application;
[0026] Figure 5 A schematic diagram of the position relationship of a binocular camera provided in an embodiment of the present application;
[0027] Figure 6 A schematic diagram of an image captured by a binocular camera provided in an embodiment of the present application;
[0028] Figure 7 A schematic diagram of the position relationship of another binocular camera provided in an embodiment of the present application;
[0029] Figure 8 A schematic diagram of images captured by a binocular camera before and after the optical image stabilization module is turned on according to an embodiment of the present application;
[0030] Fig. 9 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application;
[0031] Fig.10A schematic diagram of a software architecture of an electronic device provided in an embodiment of the present application;
[0032] Fig.11 A schematic diagram of a flow chart of a camera parameter calibration method provided in an embodiment of the present application;
[0033] Fig.12 A flowchart of a method for calculating image depth provided in an embodiment of the present application;
[0034] Fig.13 A schematic diagram of depth of each position in an image provided in an embodiment of the present application;
[0035] Fig.14 A schematic diagram of the composition of an electronic device provided in an embodiment of the present application;
[0036] Fig.15 A schematic diagram of the composition of a chip system provided in an embodiment of the present application. DETAILED DESCRIPTION
[0037] The words "first", "second" and "third" in the embodiments of the present application are used to distinguish different objects rather than to limit a specific order. In addition, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a concrete way.
[0038] To facilitate understanding, the following is an introduction to the process of imaging objects in the three-dimensional world in the camera.
[0039] Please refer to Figure 1 , is a schematic diagram of an imaging process of an object in a camera provided in an embodiment of the present application. It should be understood that Figure 1 The purpose is just to illustrate the imaging process of objects in the three-dimensional world in the camera. For the convenience of explanation, the lens 101, the photosensitive element 102, the processor 103 and the display screen 104 are all presented in a two-dimensional form. In actual applications, these elements are all three-dimensional objects.
[0040] like Figure 1As shown, the light reflected by the point P in the three-dimensional space will first pass through the lens 101 of the camera, and then be projected onto the photosensitive element 102 of the camera. The photosensitive element 102 of the camera can be CMOS (Complementary Metal Oxide Semiconductor), CCD (Charge-Coupled Device), etc. In addition, it should be noted that the optical center of the camera is the center c of the lens 101, and the focal length of the camera is the distance f between the lens 101 and the photosensitive element 102.
[0041] After receiving the light reflected from point P, the photosensitive element 102 of the camera generates image information P1 of point P, and sends the image information to the camera processor 103 for analog-to-digital conversion, that is, converting the image information from analog signals to digital signals. Finally, the camera processor 103 sends the image information converted into digital signals to the camera display screen 104 for display.
[0042] Based on the description of the above imaging process, the external and internal parameters of the camera are introduced below.
[0043] From the above imaging process of point P, it can be seen that point P undergoes multiple coordinate system transformations from three-dimensional space to display on the display screen. Figure 1 As shown, initially, point P is located in the three-dimensional space coordinate system w-kmn (also called the world coordinate system). Then, when the light reflected by point P reaches the lens 101, from the perspective of the camera, point P is located in the camera coordinate system c-xyz. Then, when the light reflected by point P reaches the photosensitive element 102, the image of point P is in the coordinate system i-jtz of the photosensitive element. Finally, the image of point P is displayed in the coordinate system upv of the display screen 104.
[0044] In the embodiment of the present application, the external parameters of the camera refer to the matrix that converts the world coordinate system to the camera coordinate system. Figure 1 , which is the matrix that can transform the coordinates of point P in the world coordinate system w-kmn to the coordinates in the camera coordinate system c-xyz.
[0045] It should be understood that the world coordinate system can be rotated and translated to coincide with the camera coordinate system, that is, converted to the camera coordinate system. Therefore, the matrix for converting the world coordinate system to the camera coordinate system may include a rotation matrix R and a translation matrix T. The rotation matrix R is a 3*3 matrix, and the translation matrix T is a 3*1 matrix.
[0046] definition Figure 1The coordinates of point P in the world coordinate system are (kw, mw, nw), and the coordinates of point P in the camera coordinate system are (xc, yc, zc). Then the coordinates of point P in the world coordinate system and the coordinates of point P in the camera coordinate system satisfy the following formula (1).
[0047]
[0048] The above formula (1) means that the coordinates (kw, mw, nw) of point P in the world coordinate system can be transformed into coordinates (xc, yc, zc) in the camera coordinate system through rotation (multiplication with the rotation matrix R) and translation (addition with the translation matrix T).
[0049] Based on the above description, it can be known that the external parameters of the camera may include the rotation matrix R and translation matrix T mentioned above.
[0050] In the embodiment of the present application, the intrinsic parameter of the camera refers to the conversion matrix that converts the camera coordinate system to the coordinate system of the display screen. Figure 1 , which is a matrix that can transform the coordinates of point P in the camera coordinate system c-xyz to the coordinate system upv of the display screen 104.
[0051] The following first describes how to calculate the coordinates of point P2 in the coordinate system i-jtz of the photosensitive element 102 based on the coordinates of point P in the camera coordinate system c-xyz.
[0052] To facilitate calculation, we establish Figure 2 The virtual photosensitive element 201 is shown. The virtual photosensitive element 201 is symmetrical with the photosensitive element 102 about the lens 101.
[0053] It should be understood that the intersection point P2 of P-P1 and the virtual photosensitive element 201 is symmetrical with P1 about the center of point c. The imaging plane and photosensitive plane involved in the subsequent embodiments of the present application all refer to the plane where the virtual photosensitive element 201 is located.
[0054] In this way, the above problem is converted into: how to calculate the coordinates of point P2 in the coordinate system i'-j't'z' of the photosensitive plane based on the coordinates of point P in the camera coordinate system c-xyz.
[0055] definition Figure 2 The coordinates of point P in the camera coordinate system are (xc, yc, zc), and the coordinates of point P2 in the camera coordinate system are (j'c, t'c, z'c). Based on the principle of similar triangles, the following formula (2) can be obtained.
[0056]
[0057] Wherein, f is the distance between the lens 101 and the photosensitive element 102, that is, the focal length of the camera. The following formula (3) can be obtained from the above formula (2).
[0058]
[0059] That is to say, when the coordinates of point P in the camera coordinate system c-xyz are known, the coordinates of point P2 in the camera coordinate system c-xyz can be obtained by the above formula (3).
[0060] Depend on Figure 2 It can be seen that the camera coordinate system c-xyz can be obtained by translating it along the z axis, and the imaging plane is a two-dimensional plane, so the coordinate system i'-j't'z of the imaging plane can be the two-dimensional coordinate system j'i't'.
[0061] Define the coordinates of point P2 in the coordinate system j'i't' of the photosensitive plane as (j'i', t'i'), then the coordinates of point P in the camera coordinate system c-xyz (xc, yc, zc) and the coordinates of point P2 in the coordinate system j'i't' of the photosensitive plane (j'i', t'i') satisfy the following formula (4).
[0062]
[0063] The above describes how to calculate the coordinates of point P2 in the coordinate system j'i't' of the photosensitive plane based on the coordinates of point P in the camera coordinate system c-xyz. From the above description, it can be seen that the transformation matrix from the camera coordinate system c-xyz to the coordinate system j'i't' of the photosensitive element 102 is K.
[0064] The following describes how to calculate the coordinates of the image of point P in the coordinate system upv of the display screen 104 based on the coordinates of point P2 in the coordinate system j'i't' of the photosensitive plane.
[0065] The origin of the coordinate system upv of the display screen 104 is at the upper left corner p of the display screen, and the coordinate unit of the coordinate system is pixel. Therefore, the coordinate system j'i't' of the photosensitive plane can be converted into the coordinate system upv of the display screen 104 after translation and unit conversion.
[0066] Define that a 1-meter imaging element can fit a pixels in the u-axis direction of the display screen, and b pixels in the v-axis direction of the display screen. The coordinate system j'i't' of the photosensitive plane is translated by c pixels along the u-axis and d pixels along the v-axis to reach the upper left corner of the image. The coordinates of point P2 in the coordinate system j'i't' of the photosensitive plane are (j'i', t'i'), and the coordinates of the image of point P in the coordinate system upv of the display screen 104 are (up, vp). Then the following formula (5) can be obtained.
[0067]
[0068] The above describes how to calculate the coordinates of the image of point P in the coordinate system upv of the display screen 104 according to the coordinates of point P2 in the coordinate system j'i't' of the photosensitive plane. From the above description, it can be seen that the transformation matrix from the coordinate system j'i't' of the photosensitive plane to the coordinate system upv of the display screen 104 is S.
[0069] The following formula (6) can be obtained from the above formula (4) and formula (5).
[0070]
[0071] Among them, SK is the transformation matrix that transforms the camera coordinate system to the coordinate system of the display screen, which is the intrinsic parameter of the camera.
[0072] The above are the specific meanings of the external parameters and internal parameters of the camera in the embodiment of the present application. From the above description, it can be seen that the internal parameters of the camera refer to the parameters related to the characteristics of the camera itself, such as focal length, pixel size, the position relationship between the optical center of the camera and the photosensitive element, etc. The external parameters of the camera refer to the parameters of the camera in the world coordinate system, such as the position of the optical center in the world coordinate system, the posture, the relative position relationship between the optical centers of the two cameras, etc.
[0073] The depth of each position in the image captured by the binocular camera can be calculated based on the internal and external parameters of the binocular camera. In the embodiment of the present application, the binocular camera refers to two cameras set in the same electronic device, such as the main camera and the secondary camera of a mobile phone. The image captured by the binocular camera may refer to the image taken by the binocular camera and / or the image previewed on the display screen of the binocular camera, which is not specifically limited here.
[0074] The specific process of calculating the depth of each position in the image captured by the binocular camera is described below.
[0075] Please refer to Figure 3 , is a schematic diagram of a flow chart of calculating the depth of each position in an image provided by an embodiment of the present application. Figure 3 As shown, the process may include the following steps.
[0076] S301 . Obtain an image I1 captured by camera e, an image I2 captured by camera f, a parameter Q of camera e, and a parameter P of camera f.
[0077] Among them, camera e and camera f are two cameras in the binocular camera. Image I1 is the image captured by camera e in a certain time period, and image I2 is the image captured by camera f in the same time period. The parameter Q of camera e includes the external parameters and internal parameters of camera e, and the parameter P of camera f includes the external parameters and internal parameters of the camera.
[0078] S302 , performing distortion correction on image I1 and image I2 respectively to obtain image I3 and image I4 .
[0079] The image I3 is the image I1 after distortion correction, and the image I4 is the image I2 after distortion correction.
[0080] The distortion correction in the embodiments of the present application may refer to the distortion correction of radial distortion. Radial distortion is caused by the fact that light rays bend farther from the center of the lens than near the center of the lens, resulting in distortion at the edge of the image. Common radial distortions include pincushion distortion and barrel distortion. Figure 4 As shown, assuming that the original image is a square grid, the pincushion distortion refers to the edge of the grid bending toward the center of the image, and the barrel distortion refers to the edge of the grid bending in the opposite direction of the image center.
[0081] OPENCV (Open Source Computer Vision Library), MATLAB TM A variety of methods for image distortion correction are provided in the literature, and this application does not specifically limit the method for distortion correction.
[0082] S303 , correcting the image I3 and the image I4 into parallel images I5 and I6 according to the parameter Q and the parameter P.
[0083] Among them, image I5 corresponds to image I3, and image I6 corresponds to image I4.
[0084] Since the imaging plane of camera e and the imaging plane of camera f may not be completely parallel, image I3 and image I4 may not be images from the same viewing angle. An example is given below.
[0085] Please refer to Figure 5 , is a schematic diagram of the position relationship of a binocular camera provided in an embodiment of the present application. Figure 5 As shown, the binocular camera includes camera e and camera f. Figure 5 It can be seen that the imaging plane of camera e is not completely parallel to the imaging plane of camera f. Figure 5 When you click on the candle in the Figure 6 Image shown.
[0086] Please refer to Figure 6 , is a schematic diagram of an image captured by a binocular camera provided in an embodiment of the present application. It can be seen that the candle image captured by camera e is a top-down perspective, while the candle image captured by camera f is an upward perspective. In other words, the images captured by camera e and camera f are not images from the same perspective.
[0087] When the images captured by the two cameras of a binocular camera are inconsistent, the depth of each position in the image cannot be directly calculated based on the two images.
[0088] In the embodiment of the present application, the images I3 and I4 can be corrected into parallel images I5 and I6 by an epipolar correction method, such as Bouguet epipolar correction. The Bouguet epipolar correction refers to rotating the imaging planes of the two cameras by half, so that the error caused by the reprojection of the left and right images is minimized and the common area is maximized.
[0089] like Figure 7 As shown in Figure 2, after Bouguet polar correction, Figure 5 The relative position relationship between camera e and camera f can be equivalent to Figure 7 As shown, the imaging plane of camera e is parallel to the imaging plane of camera f. In other words, after Bouguet epipolar correction, image I3 and image I4 can be regarded as images captured when the imaging plane of camera e is parallel to the imaging plane of camera f. This is conducive to improving the accuracy of calculating the depth of each position in the image.
[0090] It should be noted that the above S303 can only correct the two images to be nearly parallel. To further improve the accuracy, the image I5 and the image I6 can be further corrected through the following S304-S307.
[0091] S304 , performing feature point matching on image I5 and image I6 to obtain initial feature point matching sets C1 and C2 .
[0092] Among them, feature points are also called interest points or corner points. They usually have the advantages of rotation invariance, illumination invariance, and perspective invariance. They are one of the important features of images and are often used in applications such as target matching, target tracking, and 3D reconstruction. Point features mainly refer to obvious points in the image, such as prominent corner points, edge endpoints, extreme points, etc. The operator used to extract feature points is called interest point extraction (detection) operator.
[0093] Feature point matching can include detection process and matching process. Among them, the commonly used feature point detection methods include Harris corner point detection, SIFT feature detection, FAST feature detection and SURF feature detection. Among them, the corner point refers to the point with sufficiently high grayscale change value in each direction within the neighborhood, and is the point with the maximum curvature on the edge curve of the image.
[0094] Taking Harris corner detection as an example, the core of the algorithm is to use a local window to move on the image to determine whether the grayscale has changed significantly. If the grayscale value in the window (on the gradient map) has a large change, then there is a corner in the area where the window is located. Harris corner detection can include the following three steps: when the window (local area) moves in both horizontal and vertical directions at the same time, calculate the pixel value change E (x, y) inside the window; for each window, calculate its corresponding corner response function R; then perform threshold processing on the function. If R>threshold, it means that the window corresponds to a corner feature.
[0095] It should be noted that OPENCV provides a function cv2.cornerHarris for implementing Harris corner point detection, which can be directly called to detect feature points in an image.
[0096] Exemplarily, the matching process may include the following steps: establishing a feature point database of image I5, which includes the data structure of each feature point, such as position coordinates, scale, direction, feature vector, etc.; matching each feature point of image I6 in the database one by one, searching for its nearest neighbor and next nearest neighbor feature points in the database according to the Euclidean distance of the feature vector, and if (nearest neighbor distance / next nearest neighbor distance) is greater than a certain threshold, the feature matching is successful. In this way, the initial feature point matching set C1 of image I5 and the initial feature point matching set C2 of image I6 can be obtained.
[0097] S305 , screening the initial feature point matching sets C1 and C2 to obtain matching point sets C3 and C4.
[0098] Among them, C3 is the point set obtained after C1 is screened, and C4 is the point set obtained after C2 is screened.
[0099] In the embodiment of the present application, the screening method may be RANSAC (Random Sample Consensus). The algorithm can estimate the parameters of the mathematical model from a set of observed data including erroneous data in an iterative manner. Compared with the least squares method, the algorithm can obtain identification results faster and more accurately.
[0100] That is to say, S305 is a process of optimizing the data samples C1 and C2, and the optimized data C3 and C4 can better represent the feature point matching results of the images I5 and I6.
[0101] S306 , calculating the homography matrix H of the image I5 and the image I6 according to the matching point sets C3 and C4 .
[0102] The homography matrix describes the transformation relationship of coplanar points from an image captured from one perspective to an image captured from another perspective.
[0103] The matching point set C3 and the matching point set C4 are sets of feature points that match in the image I5 and the image I6.
[0104] It should be understood that, based on these feature points, the matrix required to transform image I6 to be parallel to image I5, namely, the homography matrix, can be calculated. The calculation process of the homography matrix is similar to the conversion between the above coordinate systems, and will not be repeated here.
[0105] S307 , warping the image I6 according to the homography matrix H to obtain an image I7 parallel to the image I5 .
[0106] Image warping is to rotate, translate, and perform other operations on the image I6 through the homography matrix H.
[0107] It should be understood that S307 may also be to perform image distortion on the image I5 according to the homography matrix H to obtain an image I8 parallel to the image I6. For ease of description, the subsequent steps take S307 as an example and are not described in detail.
[0108] After S307, images I5 and I7 are already very close to being completely parallel, and at this point the depth of each position in the image can be calculated.
[0109] S308 , calculating a disparity map of image I5 according to image I5 and image I7 .
[0110] In the embodiment of the present application, the disparity map of the image I5 can be calculated according to a stereo matching method or a convolutional neural network.
[0111] S309 , calculating the depth of each position in the image I5 according to the intrinsic parameters of the camera e and the disparity map of the image I5 , and obtaining the depth map of the image I5 .
[0112] Exemplarily, the relationship between the disparity g and the depth h between any two points in the image I5 may be as shown in the following formula (7).
[0113]
[0114] Where, l is the distance between the optical center of camera e and the optical center of camera f. r is the focal length of camera e.
[0115] Through the above S301-S309, the depth of each position in the image captured by the binocular camera can be calculated according to the image captured by the binocular camera and the internal and external parameters of the binocular camera. In the following embodiments, the depth of each position in the image is calculated by the above S301-S309, which will not be repeated.
[0116] With the continuous development of camera technology, more and more cameras are equipped with optical image stabilization modules. Among them, the optical image stabilization module includes a floating lens group arranged between the camera lens and the imaging element. The floating lens group can compensate for the lateral shake generated when the camera captures an image, so that the image captured by the camera has better imaging quality. Among them, the lateral shake of the camera refers to the shake parallel to the lens or the photosensitive element. Specifically, Figure 1 That is, jitter parallel to the jit plane or xcy plane.
[0117] However, when the camera turns on the optical image stabilization module to take pictures, the compensation lens group will continuously move, so the lens of the optical image stabilization module will also be laterally offset relative to the optical center of the first camera, causing the image captured by the camera, the camera's intrinsic and extrinsic parameters to change, and thus causing the camera to be unable to correctly calculate the depth of each position in the image.
[0118] For example, please refer to Figure 8 , is a schematic diagram of images captured by a binocular camera before and after the optical image stabilization module is turned on, provided in an embodiment of the present application. Figure 8 The left image in the middle shows the image captured when both camera e and camera f do not have the optical image stabilization module turned on. The right image shows the image captured when camera e has the optical image stabilization module turned on and camera f has not. It can be seen that after the optical image stabilization module of camera e is turned on, the lens of the optical image stabilization module is laterally offset to the left relative to the optical center of the first camera (the dotted line is before the offset). This will not only cause the captured image to be offset to the left (the dotted line is before the offset), but will also cause the camera's intrinsic and extrinsic parameters to change, which will cause the camera to be unable to correctly calculate the depth of each position in the image.
[0119] In order to solve the above problems, the embodiments of the present application provide a camera parameter calibration method and an electronic device, which can quickly and accurately calibrate the camera's internal and external parameters when the camera's optical image stabilization module is turned on, so as to ensure that the camera can correctly calculate the depth of each position in the image, thereby ensuring that the image has a better blurring effect and improving the user experience.
[0120] In the embodiment of the present application, the electronic device may be a portable terminal with a camera, such as a mobile phone, a tablet computer, a wearable device (such as a smart watch), a vehicle-mounted device, etc. Exemplary embodiments of the portable terminal include but are not limited to a portable terminal equipped with a camera. Or a portable terminal with other operating systems. The portable terminal may also be a laptop computer with a camera. It should also be understood that in some other embodiments, the terminal may also be a desktop computer with a camera. In addition, the camera in the embodiment of the present application refers to a camera equipped with an optical image stabilization module.
[0121] As an example, see Fig. 9 , is a schematic diagram of the structure of an electronic device 900 provided in an embodiment of the present application. The camera parameter correction method provided in an embodiment of the present application can be applied to Fig. 9 In the electronic device 900 shown.
[0122] like Fig. 9 As shown, the electronic device 900 may include a processor 901, a display screen 902, a power management module 903, a battery 904, a sensor module 905, a button 906, a camera 907, and an internal memory 908, etc.
[0123] The processor 901 may include one or more processing units, for example, the processor 901 may include an application processor (AP) 911, a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video stream codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or may be integrated in one or more processors 901.
[0124] The controller may be the nerve center and command center of the electronic device 900. The controller may generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions.
[0125] The processor 901 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 901 is a cache memory. The memory may store instructions or data that the processor 901 has just used or cyclically used. If the processor 901 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 901, and thus improves the efficiency of the system.
[0126] When the processor 901 executes the instructions stored in the memory, the electronic device 900 can execute the camera parameter calibration method provided in the embodiment of the present application.
[0127] The electronic device 900 implements the image display function through a GPU, a display screen 902, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 902 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 901 may include one or more GPUs, which execute program instructions to generate or change display information.
[0128] Display screen 902 is used to display images, video streams, etc.
[0129] The power management module 903 is used to receive input from the battery 904 and to power the processor 901, the display screen 902, the camera 907, etc. The power management module 903 can also be used to monitor parameters such as the capacity of the battery 904, the number of cycles of the battery 904, and the health status (leakage, impedance) of the battery 904. In some other embodiments, the power management module 903 can also be set in the processor 901.
[0130] The camera 907 may be equipped with an optical image stabilization module.
[0131] The internal memory 908 may be used to store computer executable program codes, which include instructions. The processor 901 executes various functional applications and data processing of the electronic device 900 by running the instructions stored in the internal memory 908 .
[0132] The internal memory 908 may also store one or more computer programs corresponding to the camera parameter calibration method provided in the embodiment of the present application.
[0133] The key 906 includes a power button, a volume button, etc. The key 906 may be a mechanical key or a touch key. The electronic device 900 may receive input from the key 906 and generate key signal input related to user settings and function control of the electronic device 900.
[0134] The sensor module 905 in the electronic device 900 may include eye movement sensors, touch sensors, pressure sensors, gyroscope sensors, air pressure sensors, magnetic sensors, acceleration sensors, distance sensors, proximity light sensors, ambient light sensors, fingerprint sensors, temperature sensors, bone conduction sensors and other components to realize the sensing and / or acquisition functions of different signals.
[0135] It should be understood that the structure illustrated in this embodiment does not constitute a specific limitation on the electronic device 900. In other embodiments, the electronic device 900 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.
[0136] Above through Fig. 9 The hardware structure of the electronic device provided in the embodiment of the present application is introduced. Taking the system as an example, the software architecture of the electronic device provided in the embodiment of the present application is exemplified.
[0137] Please refer to Fig.10 , is a schematic diagram of a software architecture of an electronic device provided in an embodiment of the present application. Fig.10 As shown, the software architecture of the electronic device 1000 can be divided into four layers, which are, from top to bottom, an application layer 1001 , a framework layer 1002 , a hardware abstraction layer 1003 and a driver layer 1004 .
[0138] The application layer 1001 may include all applications installed in the electronic device, such as a camera application.
[0139] For example, the camera application may send a start instruction to the camera in response to a click operation by the user, so as to start the camera.
[0140] The applications in the application layer 1001 are all developed based on the framework layer 1002. The framework layer 1002 includes a large number of application programming interfaces (APIs) that can be used by developers. Developers can develop various applications in the application layer 1001 by using these APIs. For example, the APIs in the framework layer 1002 may include camera management services, etc.
[0141] The hardware abstraction layer (Hardware Abstract Layer) 1003 is located between the framework layer 1002 and the driver layer 1004, and is used to abstract the hardware in the electronic device. Specifically, the hardware abstraction layer 1003 can encapsulate the driver of the driver layer 1004 into a universal interface that can be called by the framework layer 1002. These universal interfaces can be compatible with various types of hardware. In this way, the operating system of the electronic device can be hardware-independent, and the applicability of various types of hardware can be improved.
[0142] In the embodiment of the present application, the hardware abstraction layer 1003 may include a camera hardware abstraction layer, etc. The driver corresponding to the camera hardware abstraction layer is a camera driver, the corresponding hardware is a camera, and the corresponding API in the framework layer 1002 is a camera management service.
[0143] The driver layer 1004 is used to drive the corresponding hardware to work. In the embodiment of the present application, the driver layer may include a camera driver, etc. Among them, the camera driver is used to drive the camera to work.
[0144] Above through Fig.10 The software architecture of the electronic device provided in the embodiment of the present application is introduced. It should be understood that: Fig.10 The software structure shown does not constitute a specific limitation on the electronic device. In other embodiments, the electronic device may include Fig.10 There is no specific limitation on more or fewer layers.
[0145] Based on the introduction of the above application scenarios, the hardware structure and software architecture of the electronic device, the camera parameter correction method provided in the embodiment of the present application is described below.
[0146] It should be noted that the camera parameter calibration method provided in the embodiment of the present application is applied to an electronic device, which includes a first camera equipped with an optical image stabilization module. In addition, for ease of description, any image captured by the first camera when the optical image stabilization module is turned on is referred to as the first image.
[0147] Please refer to Fig.11 , is a flow chart of a camera parameter calibration method provided in an embodiment of the present application. Fig.11 As shown, the method may include the following steps.
[0148] S1101. Obtain a first time period and first parameters of a first camera.
[0149] The first time period is the exposure time period of the first camera capturing the first image. It should be understood that the process of the camera capturing the image is not completed at a certain moment, but is completed within a time period. The time period is the exposure time of the corresponding image.
[0150] In the embodiment of the present application, the first time period is the exposure time period of the camera capturing the first image. Exemplarily, the exposure time period of the first image may include the exposure start time of the first image and the exposure end time of the first image. For example, the exposure start time of the first image is t1, and the exposure end time is t2, then the first time period is [t1, t2].
[0151] The first parameter includes the internal and external parameters of the camera when the optical image stabilization module is not turned on. For the introduction of the internal and external parameters of the camera, please refer to the above embodiment, and will not be repeated here.
[0152] S1102: Acquire at least one set of lens motion data of the optical image stabilization module within a first time period.
[0153] The lens motion data is used to indicate the offset of the lens of the optical image stabilization module relative to the optical center of the first camera. It should be noted that the optical center of the first camera here refers to the optical center of the first camera when the optical image stabilization module is not turned on.
[0154] In a possible design, the amount of lens motion data acquired in the first time period may be the quotient of the first time period and the first sampling frequency. The first sampling frequency is the sampling frequency of the lens motion data driven by the optical image stabilization module. Exemplarily, the first time period is [t1, t2], and the sampling frequency of the lens motion data driven by the optical image stabilization module is z, then the amount w of lens motion data acquired in the first time period may be (t2-t1) / z. In this way, as much sample data as possible can be obtained, making the sample data more representative, thereby improving the accuracy of parameter correction.
[0155] In an embodiment of the present application, the above S1102 may specifically include the following steps: sending a first request to a first driver. The first driver is a driver for an optical image stabilization module. The first request is used to request the first driver to return at least one set of lens motion data of the optical image stabilization module within a first time period. Receive at least one set of lens motion data returned by the first driver. In other words, the lens motion data of the optical image stabilization module is collected by the driver of the optical image stabilization module, that is, the first driver. The electronic device can obtain the lens data of the optical image stabilization module conveniently and accurately through the first driver.
[0156] Exemplarily, all lens motion data of the optical image stabilization module within the first time period can be written in the form of a set, such as [[Δx1, Δy1], [Δx2, Δy2], …, [Δxw, Δyw]]. Each set of data in the set is used to indicate the offset of the lens of the optical image stabilization module relative to the optical center of the first camera at each moment in the first time period.
[0157] S1103: Perform a first process on at least one set of lens motion data to obtain at least one set of first data.
[0158] The first processing includes at least one of the following: filtering and data smoothing. Filtering refers to suppressing the noise of the image while preserving the image details as much as possible. Data smoothing refers to brushing the curve with a small window of a certain width to erase the burrs and peaks of the curve. Data smoothing can remove noise to a certain extent.
[0159] That is, the purpose of S1103 is to optimize the sample data (i.e., at least one set of lens motion data), thereby improving the representativeness of the sample data and ensuring the accuracy of parameter correction. In the embodiment of the present application, the first processing can also be other processing that can optimize the quality of the sample data, which is not specifically limited here.
[0160] S1104. Perform a second process on at least one set of first data to obtain a set of second data.
[0161] In the embodiment of the present application, the second processing may include at least any one of the following: calculating the average, calculating the median, and calculating the mode. That is, the purpose of S1104 is to obtain a group of most representative sample data based on multiple sample data.
[0162] S1105 . Calculate a parameter change of the first camera according to the second data.
[0163] S1106: Correct the first parameter according to the parameter change of the first camera.
[0164] The parameter variation of the first camera may include an intrinsic parameter variation and an extrinsic parameter variation. In a possible design, correcting the first parameter according to the parameter variation of the first camera may include: correcting the intrinsic parameter of the camera in the first parameter according to the intrinsic parameter variation of the first camera; and correcting the extrinsic parameter of the camera in the first parameter according to the extrinsic parameter variation of the first camera.
[0165] Exemplarily, the second data obtained in S1104 is defined as (Δx, Δy).
[0166] It is easy to understand that the rotation matrix R in the camera extrinsic parameters will not change, so the changes in the camera extrinsic parameters are also Δx and Δy. That is, before the lens of the optical image stabilization module is laterally offset relative to the optical center of the first camera, the camera's translation matrix T1 and after the lens of the optical image stabilization module is laterally offset relative to the optical center of the first camera, the camera's translation matrix T2 satisfies the following formula (8).
[0167]
[0168] Furthermore, before the optical center of the optical image stabilization module lens is laterally offset relative to the optical center of the first camera, the camera's intrinsic parameter matrix S1K1 and after the optical center of the optical image stabilization module lens is laterally offset relative to the optical center of the first camera, the camera's intrinsic parameter matrix S2K2 satisfies the following formula (9).
[0169]
[0170] Among them, -Δx / a, -Δy / b are the changes in the intrinsic parameters of the camera.
[0171] In this way, the calibration of the internal and external parameters when the optical image stabilization module of the camera is turned on is completed.
[0172] The camera parameter correction method provided in the embodiment of the present application can calibrate the camera's internal and external parameters when the camera's optical image stabilization module is turned on, so as to ensure that the camera can correctly calculate the depth of each position in the image, thereby ensuring that the image has a better blurring effect and improving the user experience.
[0173] After the camera's internal and external parameters are calibrated, the depth of each position in the image captured by the binocular camera can be calculated based on the calibrated parameters. The details are as follows.
[0174] The method for calculating image depth provided in the embodiment of the present application is applied to an electronic device provided with at least two cameras. Each camera may be configured with an optical image stabilization module. In the second time period, each camera captures an image.
[0175] Please refer to Fig.12 , is a flow chart of a method for calculating image depth provided in an embodiment of the present application. Fig.12 As shown, the method includes:
[0176] S1201, respectively determine whether the corresponding optical image stabilization module of each camera is turned on in the second time period. If so, execute S1202. If not, the depth of each position in each image can be directly calculated, which will not be repeated here.
[0177] As an example, the electronic device may obtain information about images captured by each camera in the second period in a camera application. The image information includes whether the optical image stabilization module is turned on when the corresponding camera captures the image.
[0178] S1202: calibrate the first parameters of the corresponding camera using a camera parameter calibration method to obtain the second parameters of the corresponding camera.
[0179] The camera parameter calibration method here is the camera parameter calibration method described in any one of the above embodiments. It should be noted that, when calibrating the first parameter of the corresponding camera through the camera parameter calibration method, the second time period can be assigned to the first time period.
[0180] S1203 , calculating the depth of each position in each image according to the first parameter and the second parameter of each camera.
[0181] Specifically, the depth of each position in each image may be calculated according to a first parameter of a camera whose optical image stabilization module is not turned on in the second time period and a second parameter of a camera whose optical image stabilization module is turned on in the second time period.
[0182] The process of calculating the depth of each position in each image using the first parameter and the second parameter of each camera may be as described in S301 - S309 , which will not be described in detail herein.
[0183] The effect of the camera parameter calibration method provided in the embodiment of the present application is described below.
[0184] In a portrait shooting scene, if the camera parameter calibration method provided in the embodiment of the present application is not used to calibrate the first parameter of the camera, the depth of each position in the image is directly calculated according to the first parameter of each camera, and the depth diagram obtained can be as follows: Fig.13 As shown in the left figure. Among them, Fig.13 The greater the grayscale, the deeper the depth.
[0185] Depend on Fig.13 As can be seen from the left picture in the figure, the depth of the portrait and the depth of the background are mixed together and cannot be clearly distinguished. In this way, when the background of the image is blurred, the portrait will also be blurred, affecting the blur effect of the image.
[0186] When the camera parameter calibration method provided in the embodiment of the present application is used to calibrate the first parameter of the camera, the depth of each position in each image is calculated according to the first parameter of the camera in which the optical image stabilization module is not turned on in the second time period and the second parameter of the camera in which the optical image stabilization module is turned on in the second time period. The depth schematic diagram obtained can be shown as follows: Fig.13 As shown in the right figure. Fig.13 As can be seen from the right picture in the figure, the depth of the person and the depth of the background can be clearly distinguished. In this way, a better blurring effect can be achieved when blurring the person in the image.
[0187] The image depth calculation method provided in the embodiment of the present application can calculate the depth of each position in the image based on the calibrated intrinsic and extrinsic parameters of the camera when the optical image stabilization module of the camera is turned on, with high accuracy.
[0188] Please refer to Fig.14 , is a schematic diagram of the composition of an electronic device 1400 provided in an embodiment of the present application. The electronic device 1400 may be any electronic device in the above examples, for example, the electronic device 1400 may be a mobile phone, a computer, etc. For example, Fig.14 As shown, the electronic device 1400 may include: a processor 1401, a memory 1402, and a camera 1403. The processor 1401 may be Figure 3The processor 301 in FIG. 1 is the same as the processor 301 in FIG. 1 ; the camera 1403 includes an optical image stabilization module. The memory 1402 is used to store computer-executable instructions. Exemplarily, in some embodiments, when the processor 1401 executes the instructions stored in the memory 1402, the electronic device 1400 can be made to perform any function of the electronic device in the above embodiments to implement any camera parameter correction method in the above examples.
[0189] It should be noted that all relevant contents of each step involved in the above method embodiment can be referred to the functional description of the corresponding functional module and will not be repeated here.
[0190] Fig.15 A schematic diagram of the composition of a chip system 1500 is shown. The chip system 1500 can be arranged in an electronic device. For example, the chip system 1500 can be arranged in a mobile phone. Exemplarily, the chip system 1500 may include: a processor 1501 and a communication interface 1502, which are used to support the electronic device to implement the functions involved in the above embodiments. In one possible design, the chip system 1500 also includes a memory for storing program instructions and data necessary for the electronic device. The chip system can be composed of chips, and can also include chips and other discrete devices. It should be noted that in some implementations of the present application, the communication interface 1502 may also be referred to as an interface circuit.
[0191] It should be noted that all relevant contents of each step involved in the above method embodiment can be referred to the functional description of the corresponding functional module and will not be repeated here.
[0192] An embodiment of the present application also provides a computer storage medium, in which computer instructions are stored. When the computer instructions are executed on an electronic device, the electronic device executes the above-mentioned related method steps to implement the method in the above-mentioned embodiment.
[0193] The embodiments of the present application also provide a computer program product. When the computer program product is run on a computer, the computer is caused to execute the above-mentioned related steps to implement the method in the above-mentioned embodiments.
[0194] In addition, an embodiment of the present application also provides a device, which may specifically be a chip, component or module, and the device may include a connected processor and memory; wherein the memory is used to store computer-executable instructions, and when the device is running, the processor may execute the computer-executable instructions stored in the memory so that the chip executes the methods in the above-mentioned method embodiments.
[0195] Among them, the electronic device, computer storage medium, computer program product or chip provided in the embodiments of the present application are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding methods provided above, and will not be repeated here.
[0196] The above mainly introduces the solution provided by the embodiment of the present application from the perspective of electronic equipment. In order to realize the above functions, it includes hardware structures and / or software modules corresponding to the execution of each function. It should be easily appreciated by those skilled in the art that, in combination with the units and algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present application.
[0197] The embodiment of the present application can divide the functional modules of the devices involved therein according to the above method examples. For example, each functional module can be divided corresponding to each function, or two or more functions can be integrated into one processing module. The above integrated modules can be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of modules in the embodiment of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation.
[0198] The functions or actions or operations or steps in the above embodiments can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using a software program, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that a computer can access or may include one or more servers, data centers and other data storage devices that can be integrated with the medium. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a DVD), or a semiconductor medium (eg, a solid state disk (SSD)).
[0199] Although the present application has been described in conjunction with specific features and embodiments thereof, it is obvious that various modifications and combinations may be made thereto without departing from the spirit and scope of the present application. Accordingly, this specification and the drawings are merely exemplary illustrations of the present application as defined by the appended claims, and are deemed to have covered any and all modifications, variations, combinations or equivalents within the scope of the present application. Obviously, those skilled in the art may make various modifications and variations to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.
Claims
1. A method for calibrating camera parameters, characterized in that: Applied to an electronic device, the electronic device includes a first camera equipped with an optical image stabilization module; the first image is any image captured by the first camera when the optical image stabilization module is turned on; the method includes: Acquire a first time period and a first parameter of the first camera; the first time period is an exposure time period for the first camera to capture the first image; the first parameter includes an internal parameter and an external parameter when the optical image stabilization module is not turned on; Acquire at least one set of lens motion data of the optical image stabilization module within the first time period; the lens motion data is used to indicate an offset of the lens of the optical image stabilization module relative to the optical center of the first camera; The first parameter is corrected according to the at least one set of lens motion data.
2. The camera parameter calibration method according to claim 1, characterized in that: The correcting the first parameter according to the at least one set of lens motion data comprises: Performing a first processing on the at least one set of lens motion data to obtain at least one set of first data; the first processing includes at least one of the following: filtering, data smoothing; The first parameter is corrected according to the at least one set of first data.
3. The camera parameter calibration method according to claim 2, characterized in that: The correcting the first parameter according to the at least one set of first data comprises: Performing a second processing on the at least one set of first data to obtain a set of second data; the second processing is any one of the following: calculating the mean, calculating the median, and calculating the mode; The first parameter is corrected according to the second data.
4. The camera parameter calibration method according to claim 3, characterized in that: The correcting the first parameter according to the second data includes: Calculate a parameter change of the first camera according to the second data; the parameter change of the first camera includes an intrinsic parameter change and an extrinsic parameter change; The first parameter is corrected according to the parameter change of the first camera.
5. The camera parameter calibration method according to any one of claims 1 to 4, characterized in that: The amount of the lens motion data in the first time period is the quotient of the first time period and a first sampling frequency; the first sampling frequency is the sampling frequency of the lens motion data by driving the optical image stabilization module.
6. The camera parameter calibration method according to any one of claims 1 to 4, characterized in that: The acquiring at least one set of lens motion data of the optical image stabilization module within the first time period includes: Sending a first request to a first driver; the first driver is a driver of the optical image stabilization module; the first request is used to request the first driver to return at least one set of lens motion data of the optical image stabilization module within the first time period; The at least one set of lens motion data returned by the first driver is received.
7. A method for calculating image depth, characterized in that: Applicable to an electronic device provided with at least two cameras; each of the cameras is equipped with an optical image stabilization module; In a second time period, each of the cameras captures an image; the method comprises: Determining respectively whether the corresponding optical image stabilization module of each camera is turned on during the second time period; If so, the first parameter of the corresponding camera is calibrated by the camera parameter calibration method according to any one of claims 1 to 6 to obtain the second parameter of the corresponding camera; wherein the first time period is the second time period; The depth of each position in each of the images is calculated according to the first parameter and the second parameter of each of the cameras.
8. The method for calculating image depth according to claim 7, characterized in that: The calculating the depth of each position in each of the images according to the first parameter and the second parameter of each of the cameras comprises: The depth of each position in each of the images is calculated according to the first parameter of the camera whose optical image stabilization module is not turned on in the second time period and the second parameter of the camera whose optical image stabilization module is turned on in the second time period.
9. An electronic device, characterized in that: The electronic device comprises: a camera equipped with an optical image stabilization module, one or more processors, and one or more memories; The one or more memories are coupled to the one or more processors; the camera is connected to the one or more processors; the one or more memories store computer instructions; When the one or more processors execute the computer instructions, the electronic device executes the camera parameter correction method described in any one of claims 1 to 6 or the image depth calculation method described in any one of claims 7 to 8.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes computer instructions. When the computer instructions are executed, the camera parameter correction method according to any one of claims 1 to 6 or the image depth calculation method according to any one of claims 7 to 8 is executed.
11. A computer program product, characterized in that The computer program product includes instructions, and when the computer program product runs on a computer, the computer can execute the camera parameter correction method according to any one of claims 1 to 6 or the image depth calculation method according to any one of claims 7 to 8 according to the instructions.
