Method, device and electronic device for implementing camera calibration

Abstract

The present application discloses a method, apparatus, and computer-readable storage medium for implementing camera calibration. The specific implementation solution is as follows: Based on the first reference pose of the rigid body relative to the fixed part and the first calculated pose of the calibration board relative to the camera to be calibrated, determine the second calculated pose of the calibration board relative to the fixed part and the third calculated pose of the camera to be calibrated relative to the rigid body, where the rigid body is connected to the fixed part and is movable relative to the fixed part, and the camera to be calibrated is installed on the rigid body; Based on the first calculated pose, the second calculated pose, and the third calculated pose, determine the fourth calculated pose of the rigid body relative to the fixed part; Use the first error between the fourth calculated pose and the first reference pose to optimize the first calculated pose to determine the optimized pose of the calibration board relative to the camera to be calibrated; Based on the optimized pose, calibrate the camera to be calibrated.

Description

Technical Field

[0001] The present disclosure relates to the field of machine vision technology, and in particular, to a method, an apparatus, and an electronic device for implementing camera calibration. Background Art

[0002] A head-mounted display device is a device that can be worn on a user's head and has a display function, such as, an augmented reality glasses, a virtual reality glasses, a mixed reality glasses, etc. Various types of head-mounted display devices usually have an image acquisition device (such as a camera) and various sensors. Based on the data collected by the above devices, a positioning algorithm is used to provide a basis for image display to improve the display effect. Usually, it is necessary to calibrate the camera to ensure the accuracy of the positioning algorithm. Summary of the Invention

[0003] Embodiments of the present disclosure provide a method, an apparatus, and an electronic device for implementing camera calibration.

[0004] According to one aspect of the present disclosure, there is provided a method for implementing camera calibration, including:

[0005] Based on a first reference pose of a rigid body relative to a fixed part, and a first calculated pose of a calibration board relative to a camera to be calibrated, determining a second calculated pose of the calibration board relative to the fixed part, and a third calculated pose of the camera to be calibrated relative to the rigid body, wherein the rigid body is connected to the fixed part and is movable relative to the fixed part, and the camera to be calibrated is mounted on the rigid body;

[0006] Based on the first calculated pose, the second calculated pose, and the third calculated pose, determining a fourth calculated pose of the rigid body relative to the fixed part;

[0007] Using a first error between the fourth calculated pose and the first reference pose to optimize the first calculated pose to determine an optimized pose of the calibration board relative to the camera to be calibrated;

[0008] Calibrating the camera to be calibrated based on the optimized pose.

[0009] According to another aspect of the present disclosure, there is provided an apparatus for implementing camera calibration, including:

[0010] A first determination module, configured to determine a second calculated pose of the calibration board relative to the fixed part, and a third calculated pose of the camera to be calibrated relative to the rigid body, based on a first reference pose of the rigid body relative to the fixed part, and a first calculated pose of the calibration board relative to the camera to be calibrated, wherein the rigid body is connected to the fixed part and is movable relative to the fixed part, and the camera to be calibrated is mounted on the rigid body;

[0011] A second determination module, configured to determine a fourth calculated pose of the rigid body relative to the fixed part based on the first calculated pose, the second calculated pose, and the third calculated pose;

[0012] A third determination module, configured to optimize the first calculated pose by using a first error between the fourth calculated pose and the first reference pose, so as to determine an optimized pose of the calibration board relative to the camera to be calibrated;

[0013] A calibration module, configured to calibrate the camera to be calibrated based on the optimized pose.

[0014] According to still another aspect of the present disclosure, there is provided a computer-readable storage medium storing a computer program for executing the method for realizing camera calibration as described above.

[0015] According to still another aspect of the present disclosure, there is provided an electronic device, including:

[0016] A processor;

[0017] A memory for storing instructions executable by the processor;

[0018] The processor is configured to read the executable instructions from the memory and execute the instructions to implement the method for realizing camera calibration as described above.

[0019] According to still another aspect of the present disclosure, there is provided a computer program product including computer program instructions, which implement the method for realizing camera calibration as described above when executed by a processor.

[0020] The technical solutions of the present disclosure will be further described in detail below with reference to the drawings and embodiments. Description of the Drawings

[0021] By describing the embodiments of the present disclosure in more detail in conjunction with the drawings, the above and other objects, features, and advantages of the present disclosure will become more obvious. The drawings are used to provide a further understanding of the embodiments of the present disclosure, and constitute a part of the specification. Together with the embodiments of the present disclosure, they are used to explain the present disclosure and do not constitute a limitation to the present disclosure. In the drawings, the same reference numerals generally represent the same components or steps.

[0022] Figure 1 It is a schematic flowchart of a method for realizing camera calibration provided by an exemplary embodiment of the present disclosure.

[0023] Figure 2 It is a schematic flowchart of a method for realizing camera calibration provided by another exemplary embodiment of the present disclosure.

[0024] Figure 3It is a schematic flowchart of a method for implementing camera calibration provided by another exemplary embodiment of the present disclosure.

[0025] Figure 4 It is a schematic structural diagram of a device for implementing camera calibration provided by an exemplary embodiment of the present disclosure.

[0026] Figure 5 It is a schematic structural diagram of a device for implementing camera calibration provided by another exemplary embodiment of the present disclosure.

[0027] Figure 6 It is a schematic structural diagram of a device for implementing camera calibration provided by yet another exemplary embodiment of the present disclosure.

[0028] Figure 7 It is a structural diagram of an electronic device provided by an exemplary embodiment of the present disclosure. Detailed Description of the Embodiment

[0029] Next, exemplary embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all of the embodiments of the present disclosure. It should be understood that the present disclosure is not limited by the exemplary embodiments described herein.

[0030] It should be noted that: unless otherwise specifically stated, the relative arrangements of components and steps, numerical expressions and values set forth in these embodiments do not limit the scope of the present disclosure.

[0031] Those skilled in the art can understand that terms such as "first", "second", etc. in the embodiments of the present disclosure are only used to distinguish different steps, devices or modules, etc., and do not represent any specific technical meaning, nor do they indicate an inevitable logical order between them.

[0032] It should also be understood that in the embodiments of the present disclosure, "a plurality of" may refer to two or more, and "at least one" may refer to one, two or more.

[0033] It should also be understood that for any component, data or structure mentioned in the embodiments of the present disclosure, without clear limitation or contrary indication in the context, it can generally be understood as one or more.

[0034] In addition, the term "and / or" in the present disclosure is only a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B may represent: A exists alone, A and B exist simultaneously, and B exists alone. In addition, the character " / " in the present disclosure generally represents an "or" relationship between the associated objects before and after.

[0035] It should also be understood that the descriptions of the various embodiments in the present disclosure emphasize the differences between the various embodiments, and the similarities or similarities therebetween can be referred to each other. For the sake of brevity, they will not be described one by one.

[0036] At the same time, it should be understood that, for the sake of convenience of description, the dimensions of the various parts shown in the drawings are not drawn in actual proportional relationship.

[0037] The following description of at least one exemplary embodiment is actually only illustrative and in no way limits the present disclosure and its application or use.

[0038] Technologies, methods, and devices known to those of ordinary skill in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and devices should be regarded as part of the specification.

[0039] It should be noted that like reference numerals and letters denote like items in the following figures, and thus, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0040] The embodiments of the present disclosure can be applied to electronic devices such as terminal devices, computer systems, servers, etc., which can operate together with many other general-purpose or special-purpose computing system environments or configurations. Examples of well-known terminal devices, computing systems, environments, and / or configurations suitable for use with terminal devices, computer systems, servers, etc. include, but are not limited to: personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, microprocessor-based systems, set-top boxes, programmable consumer electronics, network personal computers, small computer systems, large computer systems, and distributed cloud computing technology environments including any of the above systems, and so on.

[0041] Terminal devices, computer systems, servers, etc. can be described in the general context of computer system-executable instructions (such as program modules) executed by a computer system. Generally, program modules can include routines, programs, object programs, components, logics, data structures, etc., which perform specific tasks or implement specific abstract data types. The computer system / server can be implemented in a distributed cloud computing environment, where tasks are executed by remote processing devices linked through a communication network. In a distributed cloud computing environment, program modules can be located on local or remote computing system storage media including storage devices.

[0042] Exemplary method

[0043] Figure 1 It is a schematic flow diagram of a method for implementing camera calibration provided by an exemplary embodiment of the present disclosure. Figure 1The method shown may include step 110, step 120, step 130, and step 140, which will be described separately below.

[0044] Step 110: Based on the first reference pose of the rigid body relative to the fixed part and the first calculated pose of the calibration board relative to the camera to be calibrated, determine the second calculated pose of the calibration board relative to the fixed part and the third calculated pose of the camera to be calibrated relative to the rigid body. Here, the rigid body is connected to the fixed part and is movable relative to the fixed part, and the camera to be calibrated is mounted on the rigid body.

[0045] Optionally, the fixed part may be the base of the robotic arm. The base of the robotic arm is used to carry the robotic arm. The base of the robotic arm may be stationary, and the robotic arm may be movable relative to the base of the robotic arm. The rigid body may be fixed to the end of the robotic arm. For example, the rigid body may be a flange fixed to the end of the robotic arm. The flange at the end of the robotic arm may move as the whole robotic arm moves, thereby realizing the movement of the flange at the end of the robotic arm relative to the base of the robotic arm. Of course, the rigid body is not limited to the flange at the end of the robotic arm, and the rigid body may also be fixed to other parts of the robotic arm except the end.

[0046] Optionally, the camera to be calibrated may be mounted on the rigid body by screwing, clamping, or other methods well-known to those skilled in the art. In this way, when the whole robotic arm moves, the rigid body will move as the whole robotic arm moves, and the camera to be calibrated will move due to the drive of the rigid body.

[0047] Optionally, the calibration board may also be referred to as a mark board.

[0048] It should be noted that any pose involved in the embodiments of the present disclosure includes both position and orientation. Optionally, the fixed part coordinate system may be coordinate system 1, the calibration board coordinate system may be coordinate system 2, the rigid body coordinate system may be coordinate system 3, and the camera to be calibrated coordinate system may be coordinate system 4. The pose of the rigid body relative to the fixed part may be represented as 1T3 or 1 T 3 ; the pose of the calibration board relative to the camera to be calibrated may be represented as 4T2 or 4 T 2 ; the pose of the calibration board relative to the fixed part may be represented as 1T2 or 1 T 2 ; the pose of the camera to be calibrated relative to the rigid body may be represented as 3T4 or 3 T 4 .

[0049] In step 110, the first reference pose and the first calculated pose can be obtained first. The first reference pose can be the actual pose of the rigid body relative to the fixed part (assuming it is pose X). The first calculated pose can be obtained through calculation. When the actual pose of the rigid body relative to the fixed part is pose X, the first calculated pose can be the pose of the calibration board relative to the camera to be calibrated. Next, based on the obtained first reference pose and the first calculated pose, the hand-eye calibration algorithm can be used for calculation to determine the second calculated pose and the third calculated pose.

[0050] Step 120: Determine the fourth calculated pose of the rigid body relative to the fixed part based on the first calculated pose, the second calculated pose, and the third calculated pose.

[0051] In step 120, based on the first calculated pose, the second calculated pose, and the third calculated pose, the hand-eye calibration algorithm can be used for calculation to determine the fourth calculated pose.

[0052] Step 130: Optimize the first calculated pose using the first error between the fourth calculated pose and the first reference pose to determine the optimized pose of the calibration board relative to the camera to be calibrated.

[0053] It should be noted that the first reference pose can be regarded as the observed value of the pose of the rigid body relative to the fixed part, and the fourth calculated pose can be regarded as the estimated value of the pose of the rigid body relative to the fixed part. Both the estimated value and the observed value of the pose of the rigid body relative to the fixed part can be represented in the form of a matrix. In this way, the first error can be obtained by comparing and processing the two matrices.

[0054] In step 130, the first calculated pose can be used as the variable to be optimized, and a specified optimization algorithm can be used to perform optimization processing with minimizing the first error as the optimization goal. As described above, it can be considered that the first reference pose is relatively accurate, so this value is used to optimize the fourth calculated pose. During the optimization process, the values used to calculate the fourth calculated pose can also be optimized, that is, the optimized pose of the calibration board relative to the camera to be calibrated can be determined. Optionally, the specified optimization algorithm can be a non-linear optimization algorithm.

[0055] It should be pointed out that when using the specified optimization algorithm to perform optimization processing with minimizing the first error as the optimization goal, in addition to optimizing the first calculated pose, the second calculated pose and the third calculated pose can also be optimized.

[0056] Step 140: Calibrate the camera to be calibrated based on the optimized pose.

[0057] In step 140, based on the optimized pose, the internal parameters of the camera to be calibrated can be calibrated in any feasible way.

[0058] In an embodiment of the present disclosure, based on the first reference pose of the rigid body relative to the fixed part and the first calculated pose of the calibration board relative to the camera to be calibrated, the second calculated pose of the calibration board relative to the fixed part and the third calculated pose of the camera to be calibrated relative to the rigid body can be determined. And based on the first calculated pose, the second calculated pose, and the third calculated pose, the fourth calculated pose of the rigid body relative to the fixed part can be determined. In this way, the estimated value and the observed value of the pose of the rigid body relative to the fixed part are obtained. After that, by making the estimated value and the observed value of the pose of the rigid body relative to the fixed part form an error value and performing pose optimization with reference to the error amount, the optimized pose of the calibration board relative to the camera to be calibrated can be determined. Using the determined optimized pose to calibrate the camera to be calibrated can effectively improve the accuracy and reliability of the calibration result of the camera to be calibrated, thereby providing an accurate basis for the positioning algorithm that needs to use the image data collected by the camera.

[0059] Based on the embodiment shown in Figure 1 as shown in Figure 2 before step 110, the method further includes step 102 and step 104.

[0060] Step 102, controlling the camera to be calibrated to collect an image sequence of the calibration board; the camera to be calibrated collects the image sequence during the movement of the rigid body along the movement trajectory, and the movement trajectory is associated with the first pose sequence, and the first reference pose is determined based on the first pose sequence.

[0061] It should be noted that the movement trajectory can be set in advance. For example, N (N can be 8, 10, 15, 20, etc.) trajectory points can be determined in advance, and each trajectory point can be represented by a pose of the rigid body relative to the fixed part, and then the movement trajectory is determined by the N trajectory points. Optionally, the N trajectory points can all be located on the movement trajectory.

[0062] The movement trajectory is associated with the first pose sequence, which can be understood as when the rigid body moves to the positions of the N trajectory points, the pose sequence of the rigid body relative to the fixed part forms the first pose sequence.

[0063] The first reference pose is determined based on the first pose sequence, which can be understood as the first reference pose only includes the poses in the pose sequence of the rigid body relative to the fixed part when the rigid body moves to the positions of the N trajectory points, that is, only includes the poses in the first pose sequence. Or it can be understood that the first reference pose includes the poses in the pose sequence of the rigid body relative to the fixed part when the rigid body moves to the positions of the N trajectory points, and the poses in the pose sequence of the rigid body relative to the fixed part when the rigid body moves to the positions between the positions of each trajectory point, that is, includes the poses other than the first pose sequence.

[0064] It should be noted that when the camera to be calibrated captures an image sequence, the image sequence can be captured either in a static manner or in a dynamic manner.

[0065] When using the static method, the images in the image sequence can be captured by the camera to be calibrated when the rigid body is in a stationary state relative to the fixed part.

[0066] For example, when the rigid body reaches one of the N trajectory points along the motion trajectory, it can be considered that the pose of the rigid body relative to the fixed part is a certain pose in the first pose sequence. At this trajectory point, the rigid body can be controlled to stop moving, and the camera to be calibrated can be controlled to capture an image of the calibration board. After this image capture is completed, the rigid body can be controlled to continue moving along the motion trajectory. In this way, the rigid body will reach the next of the N trajectory points, and it can be considered that the pose of the rigid body relative to the fixed part is another pose in the first pose sequence. At this trajectory point, the rigid body can be controlled to stop moving again, and the camera to be calibrated can be controlled to capture an image of the calibration board again. This process continues in this way. In this way, the camera to be calibrated can capture N images corresponding one by one to the N trajectory points, and these N images can be arranged in time sequence to form the image sequence captured by the camera to be calibrated.

[0067] It can be seen that when using the static method, by making the rigid body reach a series of points in sequence and ensuring that the camera to be calibrated captures images in a timely manner each time the rigid body reaches the corresponding point, the acquisition of the image sequence can be achieved efficiently and quickly.

[0068] When using the dynamic method, the rigid body moves along the motion trajectory at a first frequency; controlling the camera to be calibrated to capture an image sequence of the calibration board includes:

[0069] Controlling the camera to be calibrated to capture images of the calibration board at a second frequency to obtain an image sequence composed of the captured images, where the second frequency is different from the first frequency.

[0070] Here, the second frequency can be either higher than or lower than the first frequency. For example, the first frequency can be 100HZ, and the second frequency can be 70HZ. Or, the first frequency can be 60HZ, and the second frequency can be 80HZ.

[0071] For example, the rigid body can move at a fixed frequency, and the camera to be calibrated can start taking pictures at a fixed another frequency synchronously from the start of the movement of the rigid body to achieve the acquisition of the image sequence. In this way, when using the dynamic method, the movement frequency of the rigid body is different from the photographing frequency of the camera to be calibrated, and the rigid body does not need to stop and wait for the camera to be calibrated to take pictures after reaching any of the N trajectory points each time, so that the acquisition of the image sequence can be achieved efficiently and quickly.

[0072] Step 104: Based on the image sequence, obtain the second pose sequence, and the first calculated pose is determined based on the second pose sequence.

[0073] In step 104, the PNP (perspective-n-point) algorithm can be used to determine the pose corresponding to each image in the image sequence. Each determined pose is the pose of the calibration board relative to the camera to be calibrated. The determined poses arranged in order can form the second pose sequence, and the first calculated pose can be determined based on the second pose sequence.

[0074] The first calculated pose is determined based on the second pose sequence, which can be understood as the first calculated pose only includes the poses in the second pose sequence. Or it can be understood that the first calculated pose includes the poses in the second pose sequence and the poses in the pose sequence of the calibration board relative to the camera to be calibrated between adjacent image acquisition operations of the camera to be calibrated, that is, it includes the poses other than the second pose sequence.

[0075] It should be noted that when optimizing the pose according to the error amount between the estimated value and the observed value of the pose of the reference rigid body relative to the fixed part, generally a large amount of sample data is required. Each group of sample data needs to include a first reference pose and a first calculated pose. In view of this, in the embodiments of the present disclosure, the first pose sequence can be obtained based on the motion trajectory, the second pose sequence can be obtained by collecting the image sequence of the calibration board by the camera to be calibrated, and a large amount of sample data can be obtained based on the first pose sequence and the second pose sequence, so as to use the large amount of sample data for pose optimization, which is beneficial to ensuring the accuracy and reliability of the calibration result of the camera to be calibrated.

[0076] In an optional example, the first pose sequence corresponds to a first frequency, and the second pose sequence corresponds to a second frequency;

[0077] In Figure 2 the embodiment shown, as Figure 3 shown, the method further includes step 106 and step 108.

[0078] Step 106: Determine the first time difference between the second pose sequence and the first pose sequence.

[0079] It should be noted that the first pose sequence may correspond to the time system of the robotic arm movement, and the second pose sequence may correspond to the time system of the camera to be calibrated for taking pictures. These two time systems may not be synchronized. For example, there may be a time difference between the initial time of the robotic arm movement and the initial time of the camera to be calibrated for taking pictures. Therefore, in step 106, the time difference between these two time systems (such as the difference between the two initial times) can be determined, and this time difference can be used as the first time difference between the second pose sequence and the first pose sequence.

[0080] Step 108: Based on the first time difference, perform the first time synchronization on the first pose sequence and the second pose sequence; the first reference pose is determined based on the first pose sequence after the first time synchronization, and the first calculated pose is determined based on the second pose sequence after the first time synchronization. The first reference pose and the first calculated pose are aligned in time.

[0081] In step 108, based on the first time difference, the time system of the robotic arm movement and the time system of the camera to be calibrated for taking pictures can be aligned in time. For example, it can be determined which second in the time system of the camera to be calibrated for taking pictures corresponds to 0 seconds in the time system of the robotic arm movement, which second in the time system of the camera to be calibrated for taking pictures corresponds to 2 seconds in the time system of the robotic arm movement, and so on.

[0082] In an optional example, the first pose sequence is as follows:

[0083] X1, X2, X3, X4, X5, X6

[0084] Among them, for the time system of the robotic arm movement, the time corresponding to X1 is 0:00 (i.e., zero second), the time corresponding to X2 is 1:00 (i.e., the 1st second), the time corresponding to X3 is 2:00, the time corresponding to X4 is 3:00, the time corresponding to X5 is 4:00, and the time corresponding to X6 is 5:00.

[0085] The second pose sequence is as follows:

[0086] Y1, Y2, Y3, Y4, Y5, Y6

[0087] Among them, for the time system of the camera to be calibrated for taking pictures, the time corresponding to Y1 is 00:00, the time corresponding to Y2 is 1:00, the time corresponding to Y3 is 2:00, the time corresponding to Y4 is 3:00, the time corresponding to Y5 is 4:00, and the time corresponding to Y6 is 5:00.

[0088] Assume that the first time difference is 1 second. When performing the first time synchronization on the first pose sequence and the second pose sequence, it can be determined that 0:00 in the time system of the robotic arm movement corresponds to 1:00 in the time system of the camera to be calibrated for taking pictures. From this, it can be further determined that X1 in the first pose sequence is aligned with Y2 in the second pose sequence in terms of time, X2 in the first pose sequence is aligned with Y3 in the second pose sequence in terms of time, X3 in the first pose sequence is aligned with Y4 in the second pose sequence in terms of time, X4 in the first pose sequence is aligned with Y5 in the second pose sequence in terms of time, and X5 in the first pose sequence is aligned with Y6 in the second pose sequence in terms of time. In this way, when taking X1 as the first reference pose, Y2 can be taken as the first calculated pose; when taking X2 as the first reference pose, Y3 can be taken as the first calculated pose; when taking X3 as the first reference pose, Y4 can be taken as the first calculated pose; and so on for the subsequent cases, which will not be elaborated here.

[0089] It can be seen that in the embodiments of the present disclosure, the time difference between different time systems is considered. By performing the first time synchronization on the first pose sequence and the second pose sequence based on this time difference, the time corresponding to each pose in the first pose sequence can be converted to the time system of the camera to be calibrated for taking pictures, or the time corresponding to each pose in the second pose sequence can be converted to the time system of the robotic arm movement, so that the first reference pose and the first calculated pose that are aligned in time can be obtained. In this way, when using the first reference pose and the first calculated pose for calibrating the camera to be calibrated, the accuracy and reliability of the calibration result can be effectively ensured.

[0090] In an optional example, determining the first time difference between the second pose sequence and the first pose sequence includes:

[0091] Extract the first attitude sequence from the first pose sequence;

[0092] Extract the second attitude sequence from the second pose sequence;

[0093] Take the attitude sequence with the higher corresponding frequency in the first attitude sequence and the second attitude sequence as the reference attitude sequence, and perform interpolation processing on the other attitude sequence in the first attitude sequence and the second attitude sequence to obtain an interpolated attitude sequence, and the frequency corresponding to the interpolated attitude sequence is the same as that of the reference attitude sequence;

[0094] Based on the reference attitude sequence and the interpolated attitude sequence, determine the first time difference between the second pose sequence and the first pose sequence.

[0095] It should be noted that each pose in the first pose sequence can include a position and an orientation. Then, the orientations can be respectively extracted from each pose in the first pose sequence, and these orientations can be arranged in order to form the first orientation sequence. In a similar manner, the second orientation sequence can be extracted from the second pose sequence. In this way, the first orientation sequence and the first pose sequence can correspond to the same frequency (i.e., both correspond to the first frequency), and the second orientation sequence and the second pose sequence can correspond to the same frequency (i.e., both correspond to the second frequency).

[0096] Next, the first frequency corresponding to the first orientation sequence can be compared with the second frequency corresponding to the second orientation sequence. If the first frequency is higher than the second frequency, the first orientation sequence can be used as the reference orientation sequence. If the first frequency is lower than the second frequency, the second orientation sequence can be used as the reference orientation sequence. For the other orientation sequence among the first orientation sequence and the second orientation sequence, an interpolation process can be performed on it to obtain an interpolated orientation sequence whose corresponding frequency is the same as that of the reference orientation sequence. For example, if the first frequency is 100 HZ and the second frequency is 70 HZ, an interpolated orientation sequence with a corresponding frequency of 100 HZ can be obtained by interpolating the second orientation sequence.

[0097] Optionally, the interpolation process in the embodiments of the present disclosure can specifically be Spherical Linear Interpolation (Slerp), and the formula involved in Slerp can be in the following form:

[0098] Slerp(q 0 ,q 1 ,t)=q 0 (q 0 -1 q 1 ) t

[0099] =q 1 (q 1 -1 q 0 ) 1-t

[0100] =(q 0 q 1 -1 ) 1-t q 1 I

[0101] =(q 1 q 0 -1 ) t q 0

[0102] Where q0 , q 1 respectively represent the starting point and the ending point of the circular arc, and t represents a parameter and is between 0 and 1.

[0103] After obtaining the interpolation pose sequence corresponding to the same frequency as the reference pose sequence, based on the reference pose sequence and the interpolation pose sequence, the first time difference between the second pose sequence and the first pose sequence can be determined. In an alternative embodiment, based on the reference pose sequence and the interpolation pose sequence, determining the first time difference between the second pose sequence and the first pose sequence includes:

[0104] Differentiating the reference pose sequence to obtain the first angular velocity sequence;

[0105] Differentiating the interpolation pose sequence to obtain the second angular velocity sequence;

[0106] Using the cross-correlation algorithm to determine the second time difference between the first angular velocity sequence and the second angular velocity sequence;

[0107] Based on the second time difference, determining the first time difference between the second pose sequence and the first pose sequence.

[0108] Here, by differentiating the reference pose sequence, the angular velocity corresponding to each pose in the reference pose sequence can be obtained, and these angular velocities arranged in order can form the first angular velocity sequence. Moreover, the first angular velocity sequence and the reference pose sequence can correspond to the same frequency. In a similar manner, the second angular velocity sequence can be obtained, the second angular velocity sequence and the interpolation pose sequence can correspond to the same frequency, and since the reference pose sequence and the interpolation pose sequence correspond to the same frequency, the first angular velocity sequence and the second angular velocity sequence will also correspond to the same frequency.

[0109] It should be noted that since the camera to be calibrated is installed on a rigid body, and the rigid body is fixed at the end of the robotic arm, it can be considered that the rigid body, the camera to be calibrated, and the end of the robotic arm form a rigid body system. Since they belong to the same rigid body system, the modulus of the angular velocity in the coordinate system established based on the rigid body (i.e., the rigid body coordinate system in the above text) and the coordinate system established based on the camera to be calibrated (i.e., the camera to be calibrated coordinate system in the above text) is the same at the same time.

[0110] Next, the cross-correlation algorithm can be used to determine the second time difference between the first angular velocity sequence and the second angular velocity sequence. For example, the difference corresponding to the maximum cross-correlation response can be determined, and this difference can be used as the second time difference. Optionally, in the process of determining the time difference using the cross-correlation algorithm, the Fourier transform can be used for accelerated calculation. After that, based on the second time difference, the first time difference between the second pose sequence and the first pose sequence can be determined.

[0111] In this implementation manner, through the differential processing of the pose sequence and combined with the use of the cross-correlation algorithm, the determination of the second time difference can be achieved efficiently and quickly, so as to determine the first time difference accordingly.

[0112] In an alternative implementation manner, based on the second time difference, determining the first time difference between the second pose sequence and the first pose sequence includes:

[0113] Based on the second time difference, perform second time synchronization on the first pose sequence and the second pose sequence;

[0114] Based on the first pose sequence after the second time synchronization, determine the second reference pose of the rigid body relative to the fixed part;

[0115] Based on the second pose sequence after the second time synchronization, determine the fifth calculated pose of the calibration board relative to the camera to be calibrated, and the second reference pose and the fifth calculated pose are aligned in time;

[0116] Based on the second reference pose and the fifth calculated pose, determine the sixth calculated pose of the rigid body relative to the fixed part;

[0117] Use the second error between the sixth calculated pose and the second reference pose to optimize the second time difference to determine the optimized time difference between the first angular velocity sequence and the second angular velocity sequence;

[0118] Take the optimized time difference as the first time difference between the second pose sequence and the first pose sequence.

[0119] It should be noted that, based on the second time difference, the method of performing second time synchronization on the first pose sequence and the second pose sequence, and determining the second reference pose and the fifth calculated pose that are aligned in time can refer to the relevant description of step 108 in the above text, and will not be elaborated here.

[0120] It should be noted that the method of determining the sixth calculated pose of the rigid body relative to the fixed part based on the second reference pose and the fifth calculated pose can refer to the description of the method of determining the fourth calculated pose of the rigid body relative to the fixed part based on the first reference position and the first calculated pose in the above text (specifically, see the descriptions of step 110 and step 120), and will not be elaborated here.

[0121] It should be noted that the second reference pose can be regarded as the observed value of the pose of the rigid body relative to the fixed part, and the sixth calculated pose can be regarded as the estimated value of the pose of the rigid body relative to the fixed part. Both the estimated value and the observed value of the pose of the rigid body relative to the fixed part can be represented in the form of a matrix. In this way, the second error can be obtained by comparing and processing the two matrices.

[0122] Since the determination of the second reference pose utilizes the first pose sequence after the second time synchronization, the determination of the fifth calculated pose utilizes the second pose sequence after the second time synchronization, and the implementation of the second time synchronization is based on the second time difference, it can be considered that the determination of both the second reference pose and the sixth calculated pose depends on the second time difference. Naturally, the second error between the sixth calculated pose and the second reference pose is also related to the second time difference. In view of this, the second time difference can be used as the variable to be optimized, and a specified optimization algorithm can be used to perform optimization processing with minimizing the second error as the optimization objective to achieve the optimization of the second time difference, thereby determining the optimized time difference between the first angular velocity sequence and the second angular velocity sequence. The determined optimized time difference can be used as the first time difference. Optionally, the specified optimization algorithm can be a non-linear optimization algorithm.

[0123] In this implementation manner, an error amount can be formed between the estimated value and the observed value of the pose of the rigid body relative to the fixed part, and the time difference can be optimized with reference to the error amount to determine the optimized time difference between the first angular velocity sequence and the second angular velocity sequence, so as to determine the first time difference accordingly. This can effectively ensure the accuracy and reliability of the determined first time difference, and thus when performing the first time synchronization based on the first time difference, the accuracy and reliability of the first time synchronization result can be ensured, further ensuring the temporal alignment between the first reference pose and the first calculated pose determined based on the first time synchronization result, and then ensuring the accuracy and reliability of the calibration result of the camera to be calibrated.

[0124] Of course, the method for determining the first time difference based on the second time difference is not limited to this. For example, it is also feasible to directly determine the second time difference as the first time difference.

[0125] In the embodiments of the present disclosure, by respectively extracting the pose sequences for the first pose sequence and the second pose sequence, and through the comparison of the frequencies corresponding to the two pose sequences, the reference pose sequence can be determined efficiently and quickly. Taking the reference pose sequence as the reference, the interpolated pose sequence can be obtained through interpolation processing, so as to efficiently and quickly determine the first time difference with reference to the reference pose sequence and the interpolated pose sequence corresponding to the same frequency.

[0126] In summary, in the embodiments of the present disclosure, the acquisition of the image sequence can be implemented in a static manner or a dynamic manner. For both the static and dynamic manners, it can be considered that the 1T3 observation value (i.e., the observation value of the pose of the rigid body relative to the fixed part) is accurate. Based on this, the intermediate values (i.e., 1T2, 4T2, 4T3) required to optimize the calculation of the 1T3 estimated value (i.e., the estimated value of the pose of the rigid body relative to the fixed part) are optimized, so as to in turn make 4T2 more accurate, and then the optimized pose of the calibration board relative to the camera to be calibrated is obtained. For the static and dynamic manners, the main difference is that when using the dynamic manner, time synchronization and interpolation processing of the pose sequence are required.

[0127] Any method for implementing camera calibration provided by the embodiments of the present disclosure can be executed by any suitable device with data processing capabilities, including but not limited to: terminal devices, servers, etc. Alternatively, any method for implementing camera calibration provided by the embodiments of the present disclosure can be executed by a processor. For example, the processor executes any method for implementing camera calibration mentioned in the embodiments of the present disclosure by calling the corresponding instructions stored in the memory. This will not be elaborated further below.

[0128] Exemplary device

[0129] Figure 4 is a schematic structural diagram of a device for implementing camera calibration provided by an exemplary embodiment of the present disclosure. Figure 4 The device shown includes a first determination module 410, a second determination module 420, a third determination module 430, and a calibration module 440.

[0130] The first determination module 410 is configured to determine a second calculated pose of the calibration board relative to the fixed part and a third calculated pose of the camera to be calibrated relative to the rigid body based on a first reference pose of the rigid body relative to the fixed part and a first calculated pose of the calibration board relative to the camera to be calibrated, where the rigid body is connected to the fixed part and is movable relative to the fixed part, and the camera to be calibrated is mounted on the rigid body.

[0131] The second determination module 420 is configured to determine a fourth calculated pose of the rigid body relative to the fixed part based on the first calculated pose, the second calculated pose, and the third calculated pose.

[0132] The third determination module 430 is configured to optimize the first calculated pose by using a first error between the fourth calculated pose and the first reference pose to determine an optimized pose of the calibration board relative to the camera to be calibrated.

[0133] The calibration module 440 is configured to calibrate the camera to be calibrated based on the optimized pose.

[0134] In an optional example, as Figure 5As shown, the device further includes:

[0135] A control module 402, configured to control the camera to be calibrated to collect an image sequence of the calibration board before determining the second calculated pose of the calibration board relative to the fixed part and the third calculated pose of the camera to be calibrated relative to the rigid body based on the first reference pose of the rigid body relative to the fixed part and the first calculated pose of the calibration board relative to the camera to be calibrated; the camera to be calibrated collects the image sequence during the movement of the rigid body along the movement trajectory, and the movement trajectory is associated with the first pose sequence, and the first reference pose is determined based on the first pose sequence;

[0136] An acquisition module 404, configured to acquire a second pose sequence based on the image sequence, and the first calculated pose is determined based on the second pose sequence.

[0137] In an optional example, the rigid body moves along the movement trajectory at a first frequency; the control module 402 is specifically configured to:

[0138] Control the camera to be calibrated to collect images of the calibration board at a second frequency to obtain an image sequence composed of the collected images, and the second frequency is different from the first frequency.

[0139] In an optional example, the first pose sequence corresponds to the first frequency, and the second pose sequence corresponds to the second frequency;

[0140] As Figure 6 As shown, the device further includes:

[0141] A fourth determination module 406, configured to determine a first time difference between the second pose sequence and the first pose sequence;

[0142] A time synchronization module 408, configured to perform first time synchronization on the first pose sequence and the second pose sequence based on the first time difference; the first reference pose is determined based on the first pose sequence after the first time synchronization, the first calculated pose is determined based on the second pose sequence after the first time synchronization, and the first reference pose and the first calculated pose are aligned in time.

[0143] In an optional example, the fourth determination module 406 includes:

[0144] A first extraction sub-module, configured to extract a first pose sequence from the first pose sequence;

[0145] A second extraction sub-module, configured to extract a second pose sequence from the second pose sequence;

[0146] A processing sub-module, configured to use the pose sequence with a higher corresponding frequency in the first pose sequence and the second pose sequence as the reference pose sequence, and perform interpolation processing on the other pose sequence in the first pose sequence and the second pose sequence to obtain an interpolated pose sequence, where the frequency corresponding to the interpolated pose sequence is the same as that of the reference pose sequence;

[0147] A determination sub-module, configured to determine a first time difference between the second pose sequence and the first pose sequence based on the reference pose sequence and the interpolated pose sequence.

[0148] In an optional example, the determination sub-module includes:

[0149] A first processing unit, configured to perform differential processing on the reference pose sequence to obtain a first angular velocity sequence;

[0150] A second processing unit, configured to perform differential processing on the interpolated pose sequence to obtain a second angular velocity sequence;

[0151] A first determination unit, configured to use a cross-correlation algorithm to determine a second time difference between the first angular velocity sequence and the second angular velocity sequence;

[0152] A second determination unit, configured to determine a first time difference between the second pose sequence and the first pose sequence based on the second time difference.

[0153] In an optional example, the second determination unit is specifically configured to:

[0154] Perform second time synchronization on the first pose sequence and the second pose sequence based on the second time difference; determine a second reference pose of the rigid body relative to the fixed part based on the first pose sequence after the second time synchronization; determine a fifth calculated pose of the calibration board relative to the camera to be calibrated based on the second pose sequence after the second time synchronization, where the second reference pose and the fifth calculated pose are aligned in time; determine a sixth calculated pose of the rigid body relative to the fixed part based on the second reference pose and the fifth calculated pose; optimize the second time difference using the second error between the sixth calculated pose and the second reference pose to determine an optimized time difference between the first angular velocity sequence and the second angular velocity sequence; use the optimized time difference as the first time difference between the second pose sequence and the first pose sequence.

[0155] In an optional example, the images in the image sequence are collected by the camera to be calibrated when the rigid body is in a stationary state relative to the fixed part.

[0156] Exemplary electronic device

[0157] Next, refer to Figure 7Describe an electronic device according to an embodiment of the present disclosure. The electronic device may be either or both of the first device and the second device, or a stand-alone device independent of them, and the stand-alone device may communicate with the first device and the second device to receive the input signals collected from them.

[0158] Figure 7 FIG. illustrates a block diagram of an electronic device 700 according to an embodiment of the present disclosure.

[0159] As Figure 7 shown, the electronic device 700 includes one or more processors 701 and a memory 702.

[0160] The processor 701 may be a central processing unit (CPU) or other forms of processing units with data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 700 to perform desired functions.

[0161] The memory 702 may include one or more computer program products, and the computer program products may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory, etc. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 701 may run the program instructions to implement the methods for camera calibration and / or other desired functions of the various embodiments of the present disclosure described above. Various contents such as input signals, signal components, noise components, etc. may also be stored in the computer-readable storage medium.

[0162] In one example, the electronic device 700 may further include: an input device 703 and an output device 704, and these components are interconnected through a bus system and / or other forms of connection mechanisms (not shown).

[0163] For example, when the electronic device 700 is the first device or the second device, the input device 703 may be a microphone or a microphone array. When the electronic device 700 is a stand-alone device, the input device 703 may be a communication network connector for receiving the input signals collected from the first device and the second device.

[0164] In addition, the input device 703 may further include, for example, a keyboard, a mouse, etc.

[0165] The output device 704 may output various information to the outside. The output device 704 may include, for example, a display, a speaker, a printer, and a communication network and its connected remote output devices, etc.

[0166] Of course, for simplicity, Figure 7 only some of the components of the electronic device 700 related to the present disclosure are shown, and components such as buses, input / output interfaces, etc. are omitted. In addition, according to specific application scenarios, the electronic device 700 may further include any other appropriate components.

[0167] Exemplary computer program product and computer-readable storage medium

[0168] In addition to the above methods and devices, an embodiment of the present disclosure may also be a computer program product, which includes computer program instructions that, when run by a processor, cause the processor to execute the steps in the method for implementing camera calibration according to various embodiments of the present disclosure described in the above "Exemplary Method" section of this specification.

[0169] The computer program product may be written in any combination of one or more programming languages for programming code to perform the operations of the embodiments of the present disclosure. The programming languages include object-oriented programming languages such as Java, C++, etc., and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code may be executed entirely on the user's computing device, partially on the user's device, executed as a stand-alone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0170] In addition, an embodiment of the present disclosure may also be a computer-readable storage medium, on which computer program instructions are stored, and when the computer program instructions are run by a processor, the processor is caused to execute the steps in the method for implementing camera calibration according to various embodiments of the present disclosure described in the above "Exemplary Method" section of this specification.

[0171] The computer-readable storage medium may adopt any combination of one or more readable media. The readable media may be a readable signal medium or a readable storage medium. The readable storage medium may, for example, include but is not limited to an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples (non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0172] The basic principles of the present disclosure have been described in conjunction with specific embodiments. However, it should be noted that the advantages, benefits, effects, etc. mentioned in the present disclosure are only examples and not limitations. It cannot be considered that these advantages, benefits, effects, etc. are essential for each embodiment of the present disclosure. Additionally, the specific details disclosed above are only for illustrative and facilitating understanding purposes, rather than limitations. The above details do not limit the present disclosure to necessarily adopting the above specific details for implementation.

[0173] Each embodiment in this specification is described in a progressive manner. The key point of each embodiment is to illustrate the differences from other embodiments. For the same or similar parts among the embodiments, reference can be made to each other. For system embodiments, since they basically correspond to method embodiments, they are described relatively simply. For relevant parts, reference can be made to the corresponding descriptions in the method embodiments.

[0174] The block diagrams of the devices, apparatuses, equipment, and systems involved in the present disclosure are only illustrative examples and do not intend to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, equipment, and systems can be connected, arranged, and configured in any way. Words such as "including", "comprising", "having", etc. are open-ended terms, meaning "including but not limited to", and can be used interchangeably with each other. The word "or" and "and" used herein refer to the word "and / or", and can be used interchangeably with each other, unless the context clearly indicates otherwise. The word "such as" used herein refers to the phrase "such as but not limited to", and can be used interchangeably with each other.

[0175] The methods and apparatuses of the present disclosure can be implemented in many ways. For example, the methods and apparatuses of the present disclosure can be implemented through software, hardware, firmware, or any combination of software, hardware, and firmware. The above order of the steps for the methods is only for illustration purposes. The steps of the methods of the present disclosure are not limited to the specific order described above, unless otherwise specifically stated. Additionally, in some embodiments, the present disclosure can also be implemented as a program recorded in a recording medium, and these programs include machine-readable instructions for implementing the methods according to the present disclosure. Therefore, the present disclosure also covers the recording medium storing the programs for executing the methods according to the present disclosure.

[0176] It should also be noted that in the apparatuses, equipment, and methods of the present disclosure, each component or each step can be decomposed and / or recombined. These decompositions and / or recombinations should be regarded as equivalent solutions of the present disclosure.

[0177] The foregoing description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present disclosure. Thus, the present disclosure is not intended to be limited to the aspects shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0178] The foregoing description has been presented for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of the present disclosure to the forms disclosed herein. Although several example aspects and embodiments have been discussed above, those skilled in the art will recognize some of their variations, modifications, alterations, additions, and subcombinations.

Claims

1. A method for implementing camera calibration, including: Based on the first reference pose of the rigid body relative to the fixed part and the first calculated pose of the calibration board relative to the camera to be calibrated, determining the second calculated pose of the calibration board relative to the fixed part and the third calculated pose of the camera to be calibrated relative to the rigid body, where the rigid body is connected to the fixed part and is movable relative to the fixed part, and the camera to be calibrated is mounted on the rigid body; the first reference pose is the actual pose of the rigid body relative to the fixed part; the first calculated pose is obtained by calculation; Based on the first calculated pose, the second calculated pose, and the third calculated pose, determining the fourth calculated pose of the rigid body relative to the fixed part; Using the first error between the fourth calculated pose and the first reference pose to optimize the first calculated pose to determine the optimized pose of the calibration board relative to the camera to be calibrated; Calibrating the camera to be calibrated based on the optimized pose.

2. The method according to claim 1, wherein, Before determining the second calculated pose of the calibration board relative to the fixed part and the third calculated pose of the camera to be calibrated relative to the rigid body based on the first reference pose of the rigid body relative to the fixed part and the first calculated pose of the calibration board relative to the camera to be calibrated, the method further includes: Controlling the camera to be calibrated to collect an image sequence of the calibration board; the camera to be calibrated collects the image sequence during the movement of the rigid body along a movement trajectory, and the movement trajectory is associated with a first pose sequence, and the first reference pose is determined based on the first pose sequence; Based on the image sequence, obtaining a second pose sequence, and the first calculated pose is determined based on the second pose sequence.

3. The method according to claim 2, wherein, The rigid body moves along the movement trajectory at a first frequency; The controlling the camera to be calibrated to collect an image sequence of the calibration board includes: Controlling the camera to be calibrated to collect images of the calibration board at a second frequency to obtain an image sequence composed of the collected images, and the second frequency is different from the first frequency.

4. The method according to claim 3, wherein, The first pose sequence corresponds to the first frequency, and the second pose sequence corresponds to the second frequency; The method further includes: Determining a first time difference between the second pose sequence and the first pose sequence; Based on the first time difference, performing first time synchronization on the first pose sequence and the second pose sequence; the first reference pose is determined based on the first pose sequence after the first time synchronization, the first calculated pose is determined based on the second pose sequence after the first time synchronization, and the first reference pose and the first calculated pose are time-aligned.

5. The method according to claim 4, wherein, The determining a first time difference between the second pose sequence and the first pose sequence includes: Extracting a first pose sequence from the first pose sequence; Extracting a second pose sequence from the second pose sequence; In the first pose sequence and the second pose sequence, use the pose sequence with the higher corresponding frequency as the reference pose sequence, and perform interpolation processing on the other pose sequence in the first pose sequence and the second pose sequence to obtain an interpolated pose sequence, where the frequency corresponding to the interpolated pose sequence is the same as that of the reference pose sequence; Based on the reference pose sequence and the interpolated pose sequence, determine a first time difference between the second pose sequence and the first pose sequence.

6. The method according to claim 5, wherein, the determining the first time difference between the second pose sequence and the first pose sequence based on the reference pose sequence and the interpolated pose sequence includes: performing differential processing on the reference pose sequence to obtain a first angular velocity sequence; performing differential processing on the interpolated pose sequence to obtain a second angular velocity sequence; using a cross-correlation algorithm to determine a second time difference between the first angular velocity sequence and the second angular velocity sequence; based on the second time difference, determine the first time difference between the second pose sequence and the first pose sequence.

7. The method according to claim 6, wherein, the determining the first time difference between the second pose sequence and the first pose sequence based on the second time difference includes: performing second time synchronization on the first pose sequence and the second pose sequence based on the second time difference; based on the first pose sequence after the second time synchronization, determine a second reference pose of the rigid body relative to the fixed part; based on the second pose sequence after the second time synchronization, determine a fifth calculated pose of the calibration board relative to the camera to be calibrated, where the second reference pose and the fifth calculated pose are aligned in time; based on the second reference pose and the fifth calculated pose, determine a sixth calculated pose of the rigid body relative to the fixed part; optimize the second time difference by using a second error between the sixth calculated pose and the second reference pose to determine an optimized time difference between the first angular velocity sequence and the second angular velocity sequence; take the optimized time difference as the first time difference between the second pose sequence and the first pose sequence.

8. The method according to claim 2, wherein, the images in the image sequence are collected by the camera to be calibrated when the rigid body is in a stationary state relative to the fixed part.

9. An apparatus for implementing camera calibration, comprising: a first determination module, configured to determine a second calculated pose of the calibration board relative to the fixed part and a third calculated pose of the camera to be calibrated relative to the rigid body based on a first reference pose of the rigid body relative to the fixed part and a first calculated pose of the calibration board relative to the camera to be calibrated, wherein the rigid body is connected to the fixed part and is movable relative to the fixed part, and the camera to be calibrated is mounted on the rigid body; the first reference pose is the actual pose of the rigid body relative to the fixed part; the first calculated pose is obtained by calculation; A second determination module, configured to determine a fourth calculated pose of the rigid body relative to the fixed part based on the first calculated pose, the second calculated pose, and the third calculated pose; A third determination module, configured to optimize the first calculated pose by using a first error between the fourth calculated pose and the first reference pose, so as to determine an optimized pose of the calibration board relative to the camera to be calibrated; A calibration module, configured to calibrate the camera to be calibrated based on the optimized pose.

10. An electronic device, comprising: a memory, configured to store a computer program product; a processor, configured to execute the computer program product stored in the memory, and when the computer program product is executed, implement the method for camera calibration according to any one of claims 1 to 8 above.

11. A computer-readable storage medium, on which computer program instructions are stored, and when the computer program instructions are executed by a processor, implement the method for camera calibration according to any one of claims 1 to 8 above.

12. A computer program product, comprising computer program instructions, and when the computer program instructions are executed by a processor, implement the method for camera calibration according to any one of claims 1 to 8 above.

Images