Camera calibration method and system, medium and electronic equipment

Abstract

The invention provides a camera calibration method and system, a medium and electronic equipment. The camera calibration method comprises the following steps: acquiring a plurality of flexible multi-plane target images; constructing a calibration plate coordinate system of the flexible multi-plane target according to the image of the flexible multi-plane target; obtaining a three-dimensional coordinate conversion relation of corresponding feature points on different plane calibration plates in a photogrammetry coordinate system according to a pose relation between the calibration plate coordinate system and the photogrammetry coordinate system; obtaining a fixed pose constraint of a camera adjustment model according to the three-dimensional coordinate conversion relation; acquiring a coplanar constraint of the camera adjustment model according to a coplanar relationship of the feature points on the same plane calibration plate; obtaining a fixed distance constraint of the camera adjustment model according to the distance between adjacent points on the plane calibration plate; constructing the camera adjustment model according to the fixed pose constraint, the coplanar constraint and the fixed distance constraint; and obtaining calibration parameters of the camera according to the camera adjustment model.

Description

Technical Field

[0001] This application belongs to the field of camera calibration technology, and relates to a camera calibration method, system, medium and electronic device. Background Technology

[0002] Current camera calibration techniques are mainly applied to short-range, small-field-of-view scenarios. However, with the improvement of industrial production levels, the demand for high-precision measurement of large objects in complex industrial environments is increasing. Camera calibration is crucial for 3D vision measurement systems, and the calibration results directly affect the measurement accuracy of the 3D vision system. Existing camera calibration methods generally suffer from low accuracy of calibration parameters. Summary of the Invention

[0003] The purpose of this application is to provide a camera calibration method, system, medium, and electronic device for improving the accuracy of camera calibration parameters.

[0004] In a first aspect, this application provides a camera calibration method, the calibration method comprising: acquiring multiple images of a flexible multi-plane target, wherein the multiple images of the flexible multi-plane target are obtained by a camera capturing images of the flexible multi-plane target at different poses, the flexible multi-plane target comprising multiple planar calibration plates, wherein the number, position, and image of feature points on each planar calibration plate are identical, and the positions and orientations of each planar calibration plate are fixed; constructing a calibration plate coordinate system of the flexible multi-plane target based on the images of the flexible multi-plane target; and determining the coordinate system between the calibration plate coordinate system and the photogrammetric coordinate system. The pose relationship is used to obtain the three-dimensional coordinate transformation relationship of corresponding feature points on different planar calibration plates in the photogrammetric coordinate system; the fixed pose constraint of the camera adjustment model is obtained according to the three-dimensional coordinate transformation relationship; the coplanar constraint of the camera adjustment model is obtained according to the coplanar relationship of feature points on the same planar calibration plate; the fixed distance constraint of the camera adjustment model is obtained according to the distance between adjacent points on the planar calibration plate; the camera adjustment model is constructed according to the fixed pose constraint, the coplanar constraint, and the fixed distance constraint; and the calibration parameters of the camera are obtained according to the camera adjustment model.

[0005] In one implementation of the first aspect, the process of obtaining the three-dimensional coordinate transformation relationship of corresponding feature points on different planar calibration plates in the photogrammetric coordinate system based on the pose relationship between the calibration plate coordinate system and the photogrammetric coordinate system includes: determining the feature points on the first calibration plate and the corresponding points of the feature points on the second calibration plate; obtaining the three-dimensional coordinates of the feature points in the first calibration plate coordinate system and the three-dimensional coordinates of the corresponding points in the second calibration plate coordinate system; obtaining the three-dimensional coordinates of the feature points and the corresponding points in the photogrammetric coordinate system; and obtaining the three-dimensional coordinate transformation relationship of corresponding feature points on different planar calibration plates in the photogrammetric coordinate system based on the three-dimensional coordinates of the feature points in the first calibration plate coordinate system, the three-dimensional coordinates of the corresponding points in the second calibration plate coordinate system, and the three-dimensional coordinates of the feature points and the corresponding points in the photogrammetric coordinate system.

[0006] In one implementation of the first aspect, the process of obtaining the fixed pose constraints of the camera adjustment model based on the three-dimensional coordinate transformation relationship includes: obtaining the feature points of the planar calibration plate and the three-dimensional coordinates of the corresponding points of the feature points on the planar calibration plate in the photogrammetric coordinate system based on the three-dimensional coordinate transformation relationship; and obtaining the fixed pose constraints between the planar calibration plates based on the three-dimensional coordinates.

[0007] In one implementation of the first aspect, the process of obtaining the coplanar constraints of the camera adjustment model based on the coplanar relationship of feature points on the same plane calibration plate includes: obtaining the three-dimensional coordinates of the feature points on the same plane calibration plate in the photogrammetric coordinate system according to the three-dimensional coordinate transformation relationship; obtaining the plane equation coefficients fitted by multiple feature points on the same plane calibration plate; and obtaining the coplanar constraints based on the plane equation coefficients and the three-dimensional coordinates of the feature points on the same plane calibration plate in the photogrammetric coordinate system.

[0008] In one implementation of the first aspect, the process of obtaining the fixed distance constraint of the camera adjustment model based on the distance between adjacent points on the same plane calibration plate includes: obtaining the fixed distance constraint based on the standard length between adjacent feature points on the same plane calibration plate and the three-dimensional coordinates of the feature points on the same plane calibration plate in the photogrammetric coordinate system.

[0009] In one implementation of the first aspect, constructing the camera adjustment model based on the pose constraint, the coplanar constraint, and the fixed distance constraint further includes: constructing the camera adjustment model based on the reprojection error of the feature points, according to the pose constraint, the coplanar constraint, and the fixed distance constraint.

[0010] In one implementation of the first aspect, the process of obtaining the calibration parameters of the camera based on the camera adjustment model includes: solving the camera adjustment model using a nonlinear optimization algorithm to obtain the calibration parameters of the camera.

[0011] Secondly, this application provides a camera calibration system, comprising: an image acquisition module for acquiring multiple images of a flexible multi-plane target, wherein the multiple images of the flexible multi-plane target are obtained by a camera capturing images of the flexible multi-plane target at different poses; the flexible multi-plane target includes multiple planar calibration plates, wherein the number, position, and image of feature points on each planar calibration plate are identical, and the positions and orientations of each planar calibration plate are fixed; a calibration plate coordinate system construction module for constructing a calibration plate coordinate system of the flexible multi-plane target based on the images of the flexible multi-plane target; and a coordinate transformation relationship acquisition module for acquiring different coordinate transformation relationships based on the pose relationship between the calibration plate coordinate system and the photogrammetric coordinate system. The system includes: a 3D coordinate transformation relationship of corresponding feature points on a planar calibration plate in the photogrammetric coordinate system; a fixed pose constraint acquisition module for acquiring fixed pose constraints of the camera adjustment model based on the 3D coordinate transformation relationship; a coplanar constraint acquisition module for acquiring coplanar constraints of the camera adjustment model based on the coplanar relationship of feature points on the same planar calibration plate; a fixed distance constraint module for acquiring fixed distance constraints of the camera adjustment model based on the distance between adjacent points on the planar calibration plate; a model construction module for constructing the camera adjustment model based on the fixed pose constraints, the coplanar constraints, and the fixed distance constraints; and a parameter acquisition module for acquiring the calibration parameters of the camera based on the camera adjustment model.

[0012] Thirdly, this application provides an electronic device, the electronic device comprising: a memory storing a computer program thereon; and a processor communicatively connected to the memory for executing the computer program to implement the above-described calibration method.

[0013] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by an electronic device, implements the above-described calibration method.

[0014] As described above, the camera calibration method, system, medium, and electronic device of this application have the following beneficial effects:

[0015] The camera calibration method provided in this application can obtain the three-dimensional coordinate transformation relationship of feature points on the planar calibration board in the photogrammetric coordinate system based on the pose relationship between the calibration board coordinate system and the photogrammetric coordinate system. Furthermore, it constructs multi-condition constraints for camera calibration and optimizes the camera adjustment model through fixed pose constraints, coplanar constraints, and fixed distance constraints, thereby improving the accuracy of camera calibration parameters. Attached Figure Description

[0016] Figure 1 The diagram shown illustrates an application scenario of the camera calibration method described in this application embodiment.

[0017] Figure 2 The diagram shown is a schematic representation of the camera calibration method described in this application embodiment.

[0018] Figure 3 The diagram shown is a schematic representation of the construction of the flexible multi-planar target described in an embodiment of this application.

[0019] Figure 4 The diagram shows the process of obtaining the three-dimensional coordinate transformation relationship as described in the embodiments of this application.

[0020] Figure 5 The diagram shows a process for obtaining fixed pose constraints as described in an embodiment of this application.

[0021] Figure 6 The diagram shown is a schematic representation of the process for obtaining coplanar constraints as described in an embodiment of this application.

[0022] Figure 7 The diagram shown is a structural schematic of the camera calibration system described in an embodiment of this application.

[0023] Figure 8 The diagram shown is a structural schematic of the electronic device described in an embodiment of this application.

[0024] Component designation explanation

[0025] 1 camera

[0026] 2. Planar calibration plate

[0027] 3 Electronic devices

[0028] 31 Memory

[0029] 32 processors

[0030] 4. Camera Calibration System

[0031] 41 Image Acquisition Module

[0032] 42. Calibration Plate Coordinate System Construction Module

[0033] 43. Coordinate Transformation Relationship Acquisition Module

[0034] 44 Fixed Pose Constraint Acquisition Module

[0035] 45. Coplanar Constraint Acquisition Module

[0036] 46 Fixed Distance Constraint Acquisition Module

[0037] 47 Model Building Module

[0038] 48 Parameter Acquisition Module

[0039] Steps S11 to S18

[0040] Steps S21 to S24

[0041] Steps S31 to S32

[0042] Steps S41 to S43 Detailed Implementation

[0043] The following specific examples illustrate the implementation of this application. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. This application can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this application. It should be noted that, unless otherwise specified, the following embodiments and features in the embodiments can be combined with each other.

[0044] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of this application. Therefore, the drawings only show the components related to this application and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0045] Current camera calibration technology is mainly applied to short-distance, small-field-of-view scenarios. However, with the improvement of industrial production levels, the demand for high-precision measurement of large objects in complex industrial environments is increasing. Camera calibration is crucial for 3D vision measurement systems, and the calibration results of camera parameters directly affect the measurement accuracy of the 3D vision system.

[0046] The current calibration method uses the method of minimizing reprojection error, specifically as follows:

[0047] Based on the pinhole camera imaging model, the collinearity equation is obtained:

[0048]

[0049] Here, x0, y0, and f are the three interior orientation elements of the camera, where x0 and y0 represent the positions of the principal point relative to the image center, and f represents the perpendicular distance from the camera's geometric center to the image plane. a1, a2, a3, b1, b2, b3, c1, c2, and c3 form a 3×3 orthogonal matrix, corresponding to the three exterior orientation elements of the image. ω,k. X S ,Y S Z S These are the three exterior orientation elements of the image, representing the coordinates of the camera's geometric center S in the photogrammetric coordinate system. Δx and Δy represent distortion compensation in the x and y directions, respectively.

[0050] The method for obtaining the reprojection error of feature points based on the collinearity equation is as follows:

[0051]

[0052] Since most of the measurement process takes place in three-dimensional space, not two-dimensional space, simply minimizing the reprojection error is not enough to obtain sufficiently accurate camera parameters.

[0053] At least in response to the above problems, the following embodiments of this application provide a camera calibration method. Figure 1 The diagram shown illustrates an application scenario of the camera calibration method provided in this application. Figure 1 As shown, two planar calibration plates form a flexible multi-planar target. By repeatedly moving the camera's pose, multiple images of the flexible multi-planar target can be acquired. Electronic devices, such as local hosts or remote servers, can use these images to analyze and construct a multi-constrained camera adjustment model to obtain precise values ​​of various camera parameters, thereby achieving high-precision camera calibration in large scenes.

[0054] The technical solutions in the embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0055] Figure 2 This is a schematic diagram illustrating the camera calibration process in one embodiment of this application. Figure 2 As shown, the calibration method includes:

[0056] S11, acquire multiple images of the flexible multi-plane target. These images are obtained by a camera capturing images of the flexible multi-plane target at different poses. The flexible multi-plane target comprises multiple planar calibration plates, each with the same number, position, and image of feature points, and the positions and orientations of the calibration plates are fixed. For details on constructing a flexible multi-plane target, please refer to [link to relevant documentation]. Figure 3 .

[0057] S12, construct the calibration plate coordinate system of the flexible multi-plane target based on the image of the flexible multi-plane target.

[0058] S13, based on the pose relationship between the calibration plate coordinate system and the photogrammetric coordinate system, obtain the three-dimensional coordinate transformation relationship of corresponding feature points on different planar calibration plates in the photogrammetric coordinate system.

[0059] S14. Obtain the fixed pose constraints of the camera adjustment model based on the three-dimensional coordinate transformation relationship.

[0060] S15, obtain the coplanar constraints of the camera adjustment model based on the coplanar relationship of feature points on the same calibration plate.

[0061] S16, Obtain the fixed distance constraint of the camera adjustment model based on the distance between adjacent points on the plane calibration plate.

[0062] S17, Construct the camera adjustment model based on the fixed pose constraint, the coplanar constraint, and the fixed distance constraint.

[0063] S18, Obtain the calibration parameters of the camera according to the camera adjustment model.

[0064] As can be seen from the above description, the camera calibration method provided in this application embodiment can obtain the three-dimensional coordinate transformation relationship of feature points on the planar calibration plate in the photogrammetric coordinate system according to the pose relationship between the calibration plate coordinate system and the photogrammetric coordinate system, and further construct multi-condition constraints for camera calibration. By fixing pose constraints, coplanar constraints, and fixing distance constraints, the camera adjustment model is optimized, thereby improving the accuracy of camera calibration parameters.

[0065] Figure 4 This diagram illustrates the process of obtaining three-dimensional coordinate transformation relationships in one embodiment of this application. Figure 4 As shown, the process of obtaining the three-dimensional coordinate transformation relationship of corresponding feature points on different planar calibration plates in the photogrammetric coordinate system based on the pose relationship between the calibration plate coordinate system and the photogrammetric coordinate system includes:

[0066] S21, determine the feature points of the first calibration plate and the corresponding points of the feature points on the second calibration plate.

[0067] For example, the photogrammetric coordinate system is O c -X c Y c Z c The calibration plate coordinate system is O wi -X wi Y wi Z wi Let i represent the number of calibration plates, i = 1, 2, 3, 4. The characteristic point of the first calibration plate is p. 1 Feature point p of the first calibration plate 1 The corresponding point on the second calibration plate is p. 2 .

[0068] S22, obtain the three-dimensional coordinates of the feature point in the first calibration plate coordinate system and the three-dimensional coordinates of the corresponding point in the second calibration plate coordinate system.

[0069] For example, feature point p 1 The three-dimensional coordinates in the first calibration plate coordinate system are: Corresponding point p 2 The three-dimensional coordinates in the second calibration plate coordinate system are:

[0070] S23, obtain the three-dimensional coordinates of the feature point and the corresponding point in the photogrammetric coordinate system.

[0071] For example, feature point p 1 The three-dimensional coordinates in the photogrammetric coordinate system are: Corresponding point p 2 The three-dimensional coordinates in the photogrammetric coordinate system are:

[0072] S24. Based on the three-dimensional coordinates of the feature point in the first calibration plate coordinate system, the three-dimensional coordinates of the corresponding point in the second calibration plate coordinate system, and the three-dimensional coordinates of the feature point and the corresponding point in the photogrammetric coordinate system, obtain the three-dimensional coordinate transformation relationship of the corresponding feature points on different plane calibration plates in the photogrammetric coordinate system.

[0073] For example, please continue reading Figure 1 According to the i-th calibration plate and the photogrammetric coordinate system O c -X c Y c Z c The pose relationship between R i ,T i We can obtain:

[0074]

[0075]

[0076] Since the first calibration plate and the second calibration plate are exactly the same, and p 1 and p 2 For the corresponding point, therefore p 1 and p 2 Their coordinates are the same in their respective calibration plate coordinate systems, represented as:

[0077]

[0078] In conclusion:

[0079] From this, we can obtain the pose relationship between the coordinates of corresponding points on different calibration plates in the photogrammetric coordinate system:

[0080]

[0081]

[0082] Through R 21 ,T 21 This allows us to obtain the three-dimensional coordinate transformation relationship of the corresponding feature points on different calibration plates in the photogrammetric coordinate system.

[0083] Figure 5 This diagram illustrates the process of obtaining fixed pose constraints in one embodiment of this application. Figure 5 As shown, the process of obtaining the fixed pose constraints of the camera adjustment model based on the three-dimensional coordinate transformation relationship includes:

[0084] S31, based on the three-dimensional coordinate transformation relationship, obtain the feature points of the plane calibration plate and the three-dimensional coordinates of the corresponding points of the feature points on the plane calibration plate in the photogrammetric coordinate system.

[0085] For example, the three-dimensional coordinates of the feature points of the i-th calibration plate and the corresponding points of the j-th calibration plate in the photogrammetric coordinate system are respectively expressed as: and

[0086] S32, obtain the fixed pose constraints between the planar calibration plates based on the three-dimensional coordinates. The fixed pose constraints can be expressed as:

[0087]

[0088]

[0089] Figure 6 This is a schematic diagram illustrating the process of obtaining coplanar constraints in one embodiment of this application. For example... Figure 6 As shown, the process of obtaining the coplanar constraints of the camera adjustment model based on the coplanar relationship of feature points on the same calibration plate includes:

[0090] S41, obtain the three-dimensional coordinates of the feature points on the same plane calibration plate in the photogrammetric coordinate system according to the three-dimensional coordinate transformation relationship.

[0091] For example, the three-dimensional coordinates of the j-th feature point on the i-th calibration plate in the photogrammetric coordinate system are represented as follows:

[0092]

[0093] S42, obtain the plane equation coefficients fitted by multiple feature points on the same plane calibration plate.

[0094] For example, the coefficients of the plane equation fitted to the feature points on the i-th calibration plate are expressed as a i ,b i ,c i .

[0095] S43, obtain coplanar constraints based on the plane equation coefficients and the three-dimensional coordinates of the feature points on the same plane calibration plate in the photogrammetric coordinate system.

[0096] For example, since the three-dimensional coordinates of feature points on the same planar calibration plate satisfy a coplanar relationship, the coplanar constraint can be expressed as:

[0097]

[0098] In one embodiment of this application, the process of obtaining the fixed distance constraint of the camera adjustment model based on the distance between adjacent points on the same plane calibration plate includes: obtaining the fixed distance constraint based on the standard length between adjacent feature points on the same plane calibration plate and the three-dimensional coordinates of the feature points on the same plane calibration plate in the photogrammetric coordinate system.

[0099] For example, since the distance between adjacent points on the same planar calibration plate is fixed and known, the error between the actual distance and the standard distance between adjacent points, i.e., the fixed distance constraint, can be expressed as:

[0100]

[0101] Among them, len st Represented as the standard length of adjacent feature points, The coordinates of the j-th feature point located on the i-th calibration plate in the photogrammetric coordinate system are expressed as follows:

[0102] In one embodiment of this application, based on the pose constraint V P The coplanar constraint V c The fixed distance constraint V dis Based on the reprojection error V of the feature points x V y The camera adjustment model is constructed as follows:

[0103]

[0104] In one embodiment of this application, the process of obtaining the calibration parameters of the camera based on the camera adjustment model includes: solving the camera adjustment model using a nonlinear optimization algorithm to obtain the calibration parameters of the camera.

[0105] For example, a planar calibration plate is used as the calibration object, and initial calibration is performed using the Zhang Zhengyou calibration plate based on the planar calibration plate to obtain the initial values ​​of the camera parameters. The camera calibration parameters are then obtained by solving the camera adjustment model using the Levenberg-Marquardt nonlinear optimization algorithm.

[0106] In summary, the camera calibration method provided in this application is based on the principles of close-range photogrammetry and incremental 3D reconstruction. It obtains the 3D coordinate transformation relationship of feature points on the planar calibration plate in the photogrammetric coordinate system based on the pose relationship between the calibration plate coordinate system and the photogrammetric coordinate system. Furthermore, it constructs multi-condition constraints for a flexible multi-plane target for camera calibration. Through fixed pose constraints, coplanar constraints, and fixed distance constraints, iterative optimization of the camera adjustment model from a 3D spatial perspective yields more accurate feature point coordinates and camera parameters, achieving high-precision camera calibration under a large field of view. Applying the camera calibration method provided in this application to the finishing project of continuously cast slabs can overcome the problem of insufficient accuracy in monocular camera calibration under a large field of view and can control the measurement error within 0.1 mm.

[0107] The scope of protection of the camera calibration method described in this application is not limited to the execution order of the steps listed in this embodiment. Any solution implemented by adding, subtracting, or replacing steps in the prior art based on the principles of this application is included within the scope of protection of this application.

[0108] This application also provides a camera calibration system that can implement the camera calibration method described in this application. However, the implementation device of the camera calibration method described in this application includes, but is not limited to, the structure of the camera calibration system listed in this embodiment. Any structural modifications and substitutions of the prior art made based on the principles of this application are included within the protection scope of this application.

[0109] Figure 7 The diagram shown is a structural schematic of a camera calibration system according to an embodiment of this application. Figure 7As shown, the calibration system 4 includes: an image acquisition module 41, a calibration board coordinate system construction module 42, a coordinate transformation relationship acquisition module 43, a fixed pose constraint acquisition module 44, a coplanar constraint acquisition module 45, a fixed distance constraint acquisition module 46, a model construction module 47, and a parameter acquisition module 48. The image acquisition module 41 is used to acquire multiple images of the flexible multi-plane target. These images are obtained by a camera capturing the flexible multi-plane target at different poses. The flexible multi-plane target includes multiple planar calibration boards, each with the same number, position, and image of feature points, and the positions and orientations of the planar calibration boards are fixed. The calibration board coordinate system construction module 42 is used to construct the calibration board coordinate system of the flexible multi-plane target based on the images of the flexible multi-plane target. The coordinate transformation relationship acquisition module 43 is used to obtain the three-dimensional coordinate transformation relationship of corresponding feature points on different planar calibration boards in the photogrammetric coordinate system based on the pose relationship between the calibration board coordinate system and the photogrammetric coordinate system. The fixed pose constraint acquisition module 44 is used to acquire the fixed pose constraints of the camera adjustment model based on the three-dimensional coordinate transformation relationship. The coplanar constraint acquisition module 45 is used to acquire the coplanar constraints of the camera adjustment model based on the coplanar relationship of feature points on the same calibration plane. The fixed distance constraint acquisition module 46 is used to acquire the fixed distance constraints of the camera adjustment model based on the distance between adjacent points on the calibration plane. The model construction module 47 is used to construct the camera adjustment model based on the fixed pose constraints, the coplanar constraints, and the fixed distance constraints. The parameter acquisition module 48 is used to acquire the calibration parameters of the camera based on the camera adjustment model.

[0110] It should be noted that, Figure 7 The modules in the camera calibration system 4 shown are... Figure 2 The steps in the camera calibration method are all corresponding and will not be repeated here.

[0111] In the embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, or methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of modules / units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or units may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection of apparatuses or modules or units may be electrical, mechanical, or other forms.

[0112] The modules / units described as separate components may or may not be physically separate. The components shown as modules / units may or may not be physical modules; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules / units can be selected to achieve the objectives of the embodiments of this application, depending on actual needs. For example, the functional modules / units in the various embodiments of this application may be integrated into one processing module, or each module / unit may exist physically separately, or two or more modules / units may be integrated into one module / unit.

[0113] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0114] This application also provides a computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements the camera calibration method provided in this application. Those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program instructing a processor. The program can be stored in a computer-readable storage medium, which is a non-transitory medium, such as random access memory, read-only memory, flash memory, hard disk, solid-state hard disk, magnetic tape, floppy disk, optical disk, and any combination thereof. The above storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., digital video disc (DVD)), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0115] This application embodiment may also provide an electronic device. Figure 8 The diagram shown is a structural schematic of an electronic device 3 according to an embodiment of this application. Figure 8 As shown, in this embodiment, the electronic device 3 includes a memory 31 and a processor 32.

[0116] The memory 31 is used to store computer programs. In some possible implementations, the memory 31 may include various media capable of storing program code, such as ROM, RAM, magnetic disk, USB flash drive, memory card, or optical disk.

[0117] In this embodiment, memory 31 may include a computer system readable medium in the form of volatile memory, such as RAM and / or cache memory. Electronic device 3 may further include other removable / non-removable, volatile / non-volatile computer system storage media. Memory 31 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of this application.

[0118] The processor 32 is connected to the memory 31 and is used to execute the computer program stored in the memory 31 so that the electronic device 3 performs the camera calibration method.

[0119] For example, processor 32 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc. In other embodiments, processor 32 may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0120] The descriptions of the processes or structures corresponding to the above figures each have their own emphasis. For parts of a process or structure that are not described in detail, please refer to the relevant descriptions of other processes or structures.

[0121] The above embodiments are merely illustrative of the principles and effects of this application and are not intended to limit this application. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of this application. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in this application should still be covered by the claims of this application.

Claims

1. A camera calibration method, characterized in that, The calibration method includes: Multiple flexible multi-plane target images are acquired by a camera capturing images of the flexible multi-plane target at different poses. The flexible multi-plane target includes multiple planar calibration plates. The number, position, and image of feature points on each planar calibration plate are the same, and the position and orientation of each planar calibration plate are fixed. Construct the calibration plate coordinate system of the flexible multi-plane target based on the image of the flexible multi-plane target; Based on the pose relationship between the calibration plate coordinate system and the photogrammetric coordinate system, obtain the three-dimensional coordinate transformation relationship of corresponding feature points on different planar calibration plates in the photogrammetric coordinate system; The fixed pose constraints of the camera adjustment model are obtained based on the three-dimensional coordinate transformation relationship. The coplanar constraints of the camera adjustment model are obtained based on the coplanar relationship of feature points on the same plane calibration plate; The fixed distance constraint of the camera adjustment model is obtained based on the distance between adjacent points on the planar calibration plate; The camera adjustment model is constructed based on the fixed pose constraint, the coplanar constraint, and the fixed distance constraint. The calibration parameters of the camera are obtained based on the camera adjustment model.

2. The calibration method according to claim 1, characterized in that, The process of obtaining the three-dimensional coordinate transformation relationship of corresponding feature points on different planar calibration plates in the photogrammetric coordinate system based on the pose relationship between the calibration plate coordinate system and the photogrammetric coordinate system includes: Determine the feature points of the first calibration plate and the corresponding points of the feature points on the second calibration plate; Obtain the three-dimensional coordinates of the feature point in the first calibration plate coordinate system and the three-dimensional coordinates of the corresponding point in the second calibration plate coordinate system; Obtain the three-dimensional coordinates of the feature points and the corresponding points in the photogrammetric coordinate system; Based on the three-dimensional coordinates of the feature point in the first calibration plate coordinate system, the three-dimensional coordinates of the corresponding point in the second calibration plate coordinate system, and the three-dimensional coordinates of the feature point and the corresponding point in the photogrammetric coordinate system, the transformation relationship of the three-dimensional coordinates of the corresponding feature points on different planar calibration plates in the photogrammetric coordinate system is obtained.

3. The calibration method according to claim 1, characterized in that, The process of obtaining the fixed pose constraints of the camera adjustment model based on the three-dimensional coordinate transformation relationship includes: Based on the three-dimensional coordinate transformation relationship, the feature points of the planar calibration plate and the three-dimensional coordinates of the corresponding points of the feature points on the planar calibration plate in the photogrammetric coordinate system are obtained. The fixed pose constraints between the planar calibration plates are obtained based on the three-dimensional coordinates.

4. The calibration method according to claim 1, characterized in that, The process of obtaining the coplanar constraints of the camera adjustment model based on the coplanar relationship of feature points on the same calibration plate includes: The three-dimensional coordinates of the feature points on the same plane calibration plate in the photogrammetric coordinate system are obtained according to the three-dimensional coordinate transformation relationship. Obtain the coefficients of the plane equation fitted by multiple feature points on the same plane calibration plate; Coplanar constraints are obtained based on the plane equation coefficients and the three-dimensional coordinates of feature points on the same plane calibration plate in the photogrammetric coordinate system.

5. The calibration method according to claim 1, characterized in that, The process of obtaining the fixed distance constraint of the camera adjustment model based on the distance between adjacent points on the planar calibration plate includes: The fixed distance constraint is obtained based on the standard length between adjacent feature points on the same plane calibration plate and the three-dimensional coordinates of the feature points on the same plane calibration plate in the photogrammetric coordinate system.

6. The calibration method according to claim 1, characterized in that, The construction of the camera adjustment model based on the pose constraints, the coplanar constraints, and the fixed distance constraints also includes: constructing the camera adjustment model based on the reprojection error of the feature points, according to the pose constraints, the coplanar constraints, and the fixed distance constraints.

7. The method according to claim 1, characterized in that, The process of obtaining the calibration parameters of the camera based on the camera adjustment model includes: solving the camera adjustment model using a nonlinear optimization algorithm to obtain the calibration parameters of the camera.

8. A camera calibration system, characterized in that, The calibration system includes: The image acquisition module is used to acquire multiple flexible multi-plane target images. The multiple flexible multi-plane target images are acquired by the camera at different poses of the flexible multi-plane target. The flexible multi-plane target includes multiple planar calibration plates. The number, position and image of feature points on each planar calibration plate are the same, and the position and orientation between each planar calibration plate are fixed. The calibration plate coordinate system construction module is used to construct the calibration plate coordinate system of the flexible multi-plane target based on the image of the flexible multi-plane target; The coordinate transformation relationship acquisition module is used to acquire the three-dimensional coordinate transformation relationship of corresponding feature points on different planar calibration plates in the photogrammetric coordinate system according to the pose relationship between the calibration plate coordinate system and the photogrammetric coordinate system. The fixed pose constraint acquisition module is used to acquire the fixed pose constraints of the camera adjustment model based on the three-dimensional coordinate transformation relationship. The coplanar constraint acquisition module is used to acquire the coplanar constraints of the camera adjustment model based on the coplanar relationship of feature points on the same plane calibration plate; The fixed distance constraint acquisition module is used to acquire the fixed distance constraints of the camera adjustment model based on the distance between adjacent points on the plane calibration plate. The model building module is used to build the camera adjustment model based on the fixed pose constraint, the coplanar constraint, and the fixed distance constraint. The parameter acquisition module is used to acquire the calibration parameters of the camera based on the camera adjustment model.

9. An electronic device, characterized in that, The electronic device includes: A memory on which computer programs are stored; A processor, communicatively connected to the memory, is used to execute the computer program to implement the calibration method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by an electronic device, it implements the calibration method as described in any one of claims 1 to 7.

Images