Extrinsic parameter calibration and plane estimation method for multiple views

Abstract

The present invention belongs to the field of computer vision, and specifically relates to a multi-view extrinsic parameter calibration and plane estimation method, which aims to solve the problems of poor reliability and complex process in the existing technology when performing camera extrinsic parameter calibration and plane estimation. The method of the present invention includes: acquiring a target image; preprocessing the target image and extracting checkerboard corner points in each preprocessed target image; based on the checkerboard corner points in each target image, jointly establishing a target equation group for each target image, and using a preset optimization algorithm to solve the target equation group to obtain the extrinsic parameters of the target camera; based on the checkerboard corner points in each target image, determining the plane equations of each undetermined plane. Based on this method, by integrating checkerboard images taken from different viewpoints, the plane equations are automatically calculated while accurately solving the camera extrinsic parameters, significantly simplifying the plane estimation process.

Description

Technical Field

[0001] The present invention belongs to the field of computer vision, and in particular relates to a multi-view oriented extrinsic parameter calibration and plane estimation method. Background Art

[0002] Camera calibration is a critical step in image processing technology, widely used in target tracking, image fusion, 3D reconstruction, and robotic navigation. The goal of camera calibration is to obtain the camera's intrinsic parameters (such as focal length and principal point) and extrinsic parameters (such as rotation and translation matrices) in order to transform points in the image coordinate system into points in the world coordinate system. These points are then used in subsequent tasks such as 3D reconstruction and SLAM.

[0003] Existing camera calibration solutions are not very effective when processing checkerboard images with large viewing angles, small image proportions, and low resolution. Taking the OpenCV library as an example, due to the sensitivity of its corner detection algorithm to image quality, some corner points are often inaccurately positioned. This lack of accuracy in corner detection directly affects the accuracy and reliability of subsequent calibration.

[0004] At the same time, in this field, existing multi-view ground plane estimation methods (such as MAGE) mostly complete plane estimation through manual point selection and point cloud reconstruction, which makes point cloud reconstruction dependent on a large number of high-quality feature points, placing high requirements on images. At the same time, the process of reconstructing point clouds based on a large number of feature points is complex and time-consuming, and requires manual annotation. The quality of feature points directly affects the reconstruction accuracy, which in turn affects the effect of plane estimation. Summary of the Invention

[0005] In order to solve the above-mentioned problems in the prior art, namely, the technical problems of poor reliability and complex process in camera extrinsic calibration and plane estimation, the first aspect of the present invention proposes a multi-view extrinsic calibration and plane estimation method, comprising:

[0006] Acquire a target image, where the target image is obtained by sequentially capturing images of a plurality of undetermined planes on which checkerboard grids are placed, using a target camera based on a plurality of preset camera positions;

[0007] Preprocessing the target image and extracting checkerboard corner points from each preprocessed target image, wherein the preprocessing includes grayscale conversion, filtering, and edge detection preprocessing;

[0008] Based on the checkerboard corner points in each target image, a target equation group for each target image is jointly established, and a preset optimization algorithm is used to solve the target equation group to obtain the extrinsic parameters of the target camera, wherein the target equation group is constructed based on the mapping relationship between the world coordinate system and the camera coordinate system of the checkerboard corner points in each target image. The preset optimization algorithm solves the minimum value of a preset objective function by iteratively updating algorithm parameters to optimize the extrinsic parameters of the target camera, and the objective function is used to indicate the residual between the predicted pixel coordinates and the actual pixel coordinates;

[0009] According to the checkerboard corner points in each target image, the plane equations of each undetermined plane are determined.

[0010] In some preferred embodiments, the method of jointly establishing a set of equations for multiple target images based on checkerboard corner points in each target image, and solving the target equations using a preset optimization algorithm to obtain the extrinsic parameters of the target camera includes:

[0011] Determine the pixel coordinates and world coordinates of the checkerboard corner points in each target image;

[0012] Determine the target equation group after the multi-graph combination according to the pixel coordinates and world coordinates of the chessboard corner points, and solve the target equation group to obtain initial extrinsic parameters;

[0013] The initial extrinsic parameters are optimized using a Levenberg-Marquardt least squares optimization algorithm to obtain the extrinsic parameters of the target camera.

[0014] In some preferred embodiments, the pixel coordinates and world coordinates of the checkerboard corner points in each target image are expressed in homogeneous coordinates as follows:

[0015] ;

[0016] The target equation group after the multi-graph combination is:

[0017] ;

[0018] Where, is the parameter to be solved, is the extrinsic parameter of the target camera.

[0019] In some preferred embodiments, when jointly determining pixel coordinates of checkerboard corner points in multiple target images and corresponding world coordinates, weights corresponding to the respective target images are set to balance the contribution of the respective target images to the joint determination result.

[0020] In some preferred embodiments, the method further comprises:

[0021] Determine the reprojection error of each undetermined plane before and after extrinsic calibration;

[0022] According to the reprojection error, it is determined whether the extrinsic parameters of the target camera meet a preset accuracy threshold.

[0023] In some preferred embodiments, the checkerboard corner points in any undetermined plane The world coordinates satisfy:

[0024] ;

[0025] passing point The normal vector is , then point The plane equation of the undetermined plane is:

[0026] ;

[0027] in, , the normal vector It is calculated from the three non-collinear corner points in the chessboard placed on the plane;

[0028] The homogeneous coordinate form of the normal vector n is , then point The normal vector n satisfies:

[0029] ;

[0030] Assume that the checkerboard corner points in another undetermined plane are ,but The world coordinates satisfy:

[0031] ;

[0032] in, Represents the position transformation matrix of a set of checkerboard corner points in the world coordinate system;

[0033] passing point Normal vector satisfy:

[0034] ;

[0035] Assumed normal vector The homogeneous coordinate form of is:

[0036] , then point The plane equation of the undetermined plane is:

[0037] .

[0038] In some preferred embodiments, determining the plane equations of each undetermined plane based on the checkerboard corner points in each target image includes:

[0039] Determine the coordinates of the checkerboard corner points in the world coordinate system as follows:

[0040] ;

[0041] Where, , is the coordinate of each point in the world coordinate system when the chessboard is placed on the kth plane;

[0042] In the chessboard corner point coordinate collection The fitting plane corresponds to a set of plane normal vectors and offsets to be solved;

[0043] exist Randomly select three non-collinear points , to construct two direction vectors to solve the plane normal vector and offset corresponding to the fitting plane, and the solution process satisfies:

[0044] ;

[0045] Where, ; ; Points The x-coordinate, y-coordinate and z-coordinate of

[0046] exist In the remaining point set, traverse each point in turn Distance from the fitting plane , and set the distance threshold to determine the Is it less than the distance threshold? is less than the distance threshold, the current point is determined is a point in the plane;

[0047] After the traversal is completed, The parameters of the fitting plane are optimized by determining the set with the largest number of interior points until the final plane equation is obtained.

[0048] In a second aspect, the present invention further proposes a multi-view extrinsic parameter calibration and plane estimation system, comprising:

[0049] A data acquisition module is used to acquire a target image, wherein the target image is obtained by sequentially capturing images of a plurality of undetermined planes on which checkerboard grids are placed using a target camera based on a plurality of preset camera positions;

[0050] A preprocessing module, configured to preprocess the target image and extract checkerboard corner points from each preprocessed target image, wherein the preprocessing includes grayscale conversion, filtering, and edge detection preprocessing;

[0051] an extrinsic parameter solving module, configured to establish a target equation group for each target image based on the checkerboard corner points in each target image, and solve the target equation group using a preset optimization algorithm to obtain the extrinsic parameters of the target camera, wherein the target equation group is constructed based on the mapping relationship between the world coordinate system and the camera coordinate system of the checkerboard corner points in each target image, and the preset optimization algorithm solves the minimum value of a preset objective function by iteratively updating algorithm parameters to optimize the extrinsic parameters of the target camera, wherein the objective function is used to indicate the residual between the predicted pixel coordinates and the actual pixel coordinates;

[0052] The plane estimation module is used to determine the plane equations of each undetermined plane based on the checkerboard corner points in each target image.

[0053] In a third aspect, the present invention further provides an electronic device, comprising:

[0054] at least one processor; and

[0055] a memory communicatively connected to at least one of the processors; wherein,

[0056] The memory stores instructions that can be executed by the processor, and the instructions are used to be executed by the processor to implement the multi-view oriented extrinsic parameter calibration and plane estimation method as described in the first aspect.

[0057] In a fourth aspect, the present invention further proposes a computer-readable storage medium, characterized in that the computer-readable storage medium stores computer instructions, and the computer instructions are used to be executed by the computer to implement the multi-view extrinsic parameter calibration and plane estimation method as described in the first aspect.

[0058] Beneficial effects of the present invention:

[0059] (1) Compared with the traditional single-image calibration method, the method of the present application enhances the robustness of the calibration process by combining the corner points of a multi-view chessboard. In practical applications, images may be affected by a variety of factors such as large angles of view, small proportions, and low resolution. The method of the present application uses multi-image joint calibration, which is equivalent to combining multiple chessboards into a large chessboard, increasing the number of corner points, expanding the coverage area, and enriching the perspective mode. It effectively handles the impact of these factors and makes the calibration results more stable and reliable. It can even assign different weights according to the different locations of the chessboard in the camera imaging area (center or edge), which can alleviate the impact of imaging quality to a certain extent.

[0060] (2) Compared with traditional calibration methods, the method of this application separates the calibration process of internal and external parameters. Generally speaking, the internal parameters of a camera are fixed after leaving the factory, so they can be accurately calibrated in a laboratory environment. The method of this application calibrates the external parameters under the condition that the internal parameters are known, which can eliminate the error interference of the internal parameter calibration. At the same time, the traditional calibration method actually only uses one image in the process of solving the camera's external parameters. This method can further improve the accuracy of the external parameter calibration by using multiple checkerboard images with different perspectives.

[0061] (3) Reduce manual annotation and reliance on feature points. Traditional plane estimation usually relies on feature point matching to construct a point cloud. The quality and quantity of feature points significantly affect the performance of the algorithm. The checkerboard method directly provides high-quality feature points through geometric structure, without the need for point cloud reconstruction or time-consuming matching. This method combines the calibration and plane estimation steps into one, fully automated process, and avoids the manual annotation process. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Other features, objects and advantages of the present application will become more apparent upon reading the detailed description of non-limiting embodiments made with reference to the following drawings:

[0063] Figure 1 1 is a flow chart of a multi-view extrinsic parameter calibration and plane estimation method proposed in an embodiment of the present invention;

[0064] Figure 2 It is a structural diagram of a computer system of a server proposed in an embodiment of the present invention for implementing the method and system embodiments of the present application. DETAILED DESCRIPTION

[0065] The present application will be further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are intended only to illustrate the relevant invention and are not intended to limit the invention. It should also be noted that, for ease of description, only portions relevant to the relevant invention are shown in the accompanying drawings.

[0066] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in this application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0067] Reference Figure 1 The first embodiment of the present invention provides a multi-view extrinsic parameter calibration and plane estimation method, comprising:

[0068] Step S10, acquiring a target image, wherein the target image is obtained by sequentially capturing images of a plurality of undetermined planes on which checkerboard grids are placed, using a target camera based on a plurality of preset camera positions;

[0069] In this embodiment, the target image refers to a plurality of checkerboard images obtained by sequentially placing checkerboards for extrinsic parameter calibration on different planes and photographing the checkerboards from a plurality of fixed preset positions using a target camera with undetermined extrinsic parameters.

[0070] It should be noted that, in this embodiment, based on the division of extrinsic parameter calibration and plane estimation, the above-mentioned target cameras at the execution entity level include at least: a single target camera for extrinsic parameter calibration, and multiple target cameras for plane estimation. This embodiment does not further name and distinguish the number of target cameras, that is, the target cameras in this embodiment are used to represent all cameras to be used for subsequent camera calibration and plane estimation.

[0071] Based on this understanding, the multiple preset positions in this embodiment refer to pre-set, fixed camera shooting positions. Specifically, for camera extrinsic calibration, the multiple preset positions can be regarded as corresponding one-to-one to each target camera to be calibrated. Each target camera to be calibrated is fixed at its corresponding position and takes multiple shots when the checkerboard moves; for plane estimation, the multiple preset positions can be regarded as a sequence set corresponding to the fixed position of each target camera.

[0072] In this embodiment, it is assumed that the size of the checkerboard and the positions of each corner point in the real world are known. Therefore, this embodiment obtains multiple samples for camera extrinsic calibration by taking multiple images of the same checkerboard at different perspectives. These samples are used in subsequent steps to enhance the robustness of the calibration process.

[0073] Step S20, preprocessing the target image and extracting checkerboard corner points from each preprocessed target image, wherein the preprocessing includes grayscale conversion, filtering, and edge detection preprocessing;

[0074] In this embodiment, the purpose of preprocessing the acquired target image is to improve the image quality so as to facilitate the subsequent corner point extraction. The preprocessing in this embodiment includes three parts: grayscale processing, filtering processing and edge detection preprocessing;

[0075] Grayscale processing is used to convert color images into grayscale images to simplify image data and reduce the amount of calculation when extracting checkerboard corners, while retaining sufficient structural information for corner detection;

[0076] Filtering is used to filter grayscale images to remove noise and spots, while making the processed image smoother, which is helpful for subsequent edge detection and corner location.

[0077] Edge detection is used to identify edge information in the image using a preset edge detection algorithm (this embodiment does not limit the specific algorithm). It is easy to understand that since the corners of the checkerboard grid are generally located at the intersection of the edges, edge detection can effectively reduce the computational effort required for corner extraction.

[0078] Step S30, based on the checkerboard corner points in each target image, jointly establish a target equation group for each target image, and use a preset optimization algorithm to solve the target equation group to obtain the extrinsic parameters of the target camera, wherein the target equation group is constructed based on the mapping relationship between the world coordinate system and the camera coordinate system of the checkerboard corner points in each target image, and the preset optimization algorithm solves the minimum value of a preset objective function by iteratively updating algorithm parameters to optimize the extrinsic parameters of the target camera, and the objective function is used to indicate the residual between the predicted pixel coordinates and the actual pixel coordinates;

[0079] The preset optimization algorithm in this embodiment may be the Levenberg-Marquardt least squares optimization algorithm, which combines the advantages of gradient descent and Gauss-Newton methods to solve nonlinear least squares problems in the field. Nonlinear least squares problems are generally formulated as finding a parameter vector x such that the sum of the squared errors between the model's predicted values ​​and the observed values ​​is minimized.

[0080] Specifically, the above process includes:

[0081] Determine the pixel coordinates and world coordinates of the checkerboard corner points in each target image;

[0082] Determine the target equation group after the multi-graph combination according to the pixel coordinates and world coordinates of the chessboard corner points, and solve the target equation group to obtain initial extrinsic parameters;

[0083] The initial extrinsic parameters are optimized using a Levenberg-Marquardt least squares optimization algorithm to obtain the extrinsic parameters of the target camera.

[0084] Furthermore, it is known that each image includes:

[0085] ;

[0086] In each image, the pixel coordinates of the checkerboard corners and the world coordinates (both expressed in homogeneous coordinates) are:

[0087] ;

[0088] Let the camera intrinsic parameter be M, and the world coordinate system of the checkerboard corner point corresponding to image1 be the reference world coordinate system. The relationship between the world coordinates of the remaining points and the world coordinates of the checkerboard corner point corresponding to image1 can be expressed as:

[0089] ;

[0090] in, Represents the position transformation of the checkerboard corner points in the world coordinate system, satisfying:

[0091] ;

[0092] in is the rotation matrix variable introduced, with three degrees of freedom, which can be simplified into three variables by Rodriguez. is the displacement variable, with three degrees of freedom.

[0093] Finally, the system of equations after multiple graphs can be expressed as:

[0094] ;

[0095] Where, is the parameter to be solved, is the extrinsic parameter of the target camera.

[0096] In this embodiment, when the pixel coordinates of the checkerboard corner points in multiple target images and the corresponding world coordinates are combined, the contribution value of each target image to the combined result is balanced by setting the corresponding weight of each target image.

[0097] Compared to traditional single-image calibration methods, this method enhances the robustness of the calibration process by combining multi-view checkerboard corner points. In practical applications, images may be affected by a variety of factors such as wide viewing angles, small image size, and low resolution. This method effectively addresses the impact of these factors through multi-image joint calibration, making the calibration results more stable and reliable.

[0098] Step S40 , determining the plane equations of each undetermined plane according to the checkerboard corner points in each target image.

[0099] It is easy to understand that in the existing technology, the extrinsic parameter calibration and plane estimation solutions for multiple views generally include: 1. calibrating extrinsic parameters; 2. manually marking points on the ground / multi-plane at each viewpoint; 3. establishing the equations of the ground / multi-plane. This approach is relatively cumbersome because it requires manual marking of points after the extrinsic parameter calibration is completed;

[0100] In this embodiment, multi-image joint calibration is performed when calibrating external parameters (as described above). During calibration, the checkerboard grids are placed on different planes. Once the calibration is completed, the equations of each plane can be obtained at the same time, eliminating the need for manual point marking.

[0101] That is, this embodiment only requires a calibration process to obtain the camera extrinsic parameters and plane equations, which simplifies the traditional process.

[0102] In this embodiment, the plane equation of the undetermined plane satisfies:

[0103] ;

[0104] Where, , the normal vector It is calculated from the three non-collinear corner points in the checkerboard placed on the plane.

[0105] This embodiment provides two implementation methods for determining the plane equations of each undetermined plane. On the one hand, the plane equations can be calculated using the world coordinate system and the transformation matrix T of different planes, which is the basic scheme of this embodiment; on the other hand, in order to eliminate the interference of noise and outliers, the three-dimensional coordinates of the checkerboard corner points and the RANSAC algorithm can be used to calculate the plane equations, which is another optional scheme of this embodiment.

[0106] In the first aspect, the plane equations are calculated using the world coordinate system and the transformation matrix T of different planes, including:

[0107] Assume that the corner points of the chessboard are placed on the ground The world coordinates are:

[0108] ;

[0109] passing point The normal vector is , then point The plane equation of the undetermined plane is:

[0110] ;

[0111] in, , the normal vector It is calculated from the three non-collinear corner points in the chessboard placed on the plane;

[0112] The homogeneous coordinate form of the normal vector n is ,point With the vector n satisfying:

[0113] ;

[0114] Assume that the checkerboard corner points in another undetermined plane are , The world coordinates satisfy:

[0115] ;

[0116] in, Represents the position transformation matrix of a set of checkerboard corner points in the world coordinate system;

[0117] passing point Normal vector satisfy:

[0118] ;

[0119] Assumed normal vector The homogeneous coordinate form of is:

[0120] , then point The plane equation of the undetermined plane is:

[0121] .

[0122] Similarly, transformations on other planes can be deduced in the same way.

[0123] In this embodiment, the plane equation is calculated using the three-dimensional coordinates of the checkerboard corner points and the RANSAC algorithm, specifically including:

[0124] Determine the coordinates of the checkerboard corner points in the world coordinate system as follows:

[0125] ;

[0126] Where, , is the coordinate of each point in the world coordinate system when the chessboard is placed on the kth plane;

[0127] In the chessboard corner point coordinate collection The fitting plane corresponds to a set of plane normal vectors and offsets to be solved;

[0128] exist Randomly select three non-collinear points , to construct two direction vectors to solve the plane normal vector and offset corresponding to the fitting plane, and the solution process satisfies:

[0129] ;

[0130] Where, ; ; Points The x-coordinate, y-coordinate and z-coordinate of

[0131] exist In the remaining point set, traverse each point in turn Distance from the fitting plane , and set the distance threshold to determine the Is it less than the distance threshold? is less than the distance threshold, the current point is determined is a point in the plane;

[0132] After the traversal is completed, The parameters of the fitting plane are optimized by determining the set with the largest number of interior points until the final plane equation is obtained.

[0133] Based on the above embodiments, the present application proposes a multi-view extrinsic parameter calibration and plane estimation method. This method integrates checkerboard images taken from different perspectives to construct a set of parallel equations to accurately solve the camera extrinsic parameters. Compared with the existing technology, this method uses multi-view data to enhance the robustness of the calibration process and improves the calibration accuracy through joint solution. At the same time, the present invention also proposes a multi-view plane estimation method based on a checkerboard. By placing a checkerboard on a plane and combining multi-view corner point information, it can not only automatically calculate the equation of the plane, but also provide a high-precision geometric reference, thereby significantly simplifying the plane estimation process. This method reduces the reliance on feature point matching, avoids the tedious process of manually marking ground areas, improves the automation and efficiency of plane estimation, and enhances robustness in complex scenes. By calibrating the plane with a checkerboard, not only a stable geometric benchmark is provided for camera extrinsic parameter calibration, but also a solid foundation for subsequent detection and tracking tasks.

[0134] A second embodiment of the present application provides a multi-view extrinsic parameter calibration and plane estimation system, including:

[0135] A data acquisition module is used to acquire a target image, wherein the target image is obtained by sequentially capturing images of a plurality of undetermined planes on which checkerboard grids are placed using a target camera based on a plurality of preset camera positions;

[0136] A preprocessing module, configured to preprocess the target image and extract checkerboard corner points from each preprocessed target image, wherein the preprocessing includes grayscale conversion, filtering, and edge detection preprocessing;

[0137] an extrinsic parameter solving module, configured to establish a target equation group for each target image based on the checkerboard corner points in each target image, and solve the target equation group using a preset optimization algorithm to obtain the extrinsic parameters of the target camera, wherein the target equation group is constructed based on the mapping relationship between the world coordinate system and the camera coordinate system of the checkerboard corner points in each target image, and the preset optimization algorithm solves the minimum value of a preset objective function by iteratively updating algorithm parameters to optimize the extrinsic parameters of the target camera, wherein the objective function is used to indicate the residual between the predicted pixel coordinates and the actual pixel coordinates;

[0138] The plane estimation module is used to determine the plane equations of each undetermined plane based on the checkerboard corner points in each target image.

[0139] A third embodiment of the present application provides an electronic device, including:

[0140] at least one processor; and

[0141] a memory communicatively connected to at least one of the processors; wherein,

[0142] The memory stores instructions that can be executed by the processor, and the instructions are used to be executed by the processor to implement the above-mentioned multi-view oriented extrinsic parameter calibration and plane estimation method.

[0143] A fourth embodiment of the present application provides a computer-readable storage medium, which stores computer instructions. The computer instructions are used to be executed by the computer to implement the above-mentioned multi-view extrinsic parameter calibration and plane estimation method.

[0144] Reference below Figure 2 , which shows a structural diagram of a computer system of a server suitable for implementing the method and system embodiments of the present application. Figure 2 The server shown is merely an example and should not limit the functions and scope of use of the embodiments of the present application.

[0145] like Figure 2 As shown, the computer system includes a central processing unit (CPU) 301, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 302 or the program loaded from the storage part 308 to the random access memory (RAM) 303. Various programs and data required for system operation are also stored in the RAM 303. The CPU 301, ROM 302 and RAM 303 are connected to each other via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0146] The following components are connected to the I / O interface 305: an input section 306 including a keyboard, a mouse, and the like; an output section 307 including devices such as a cathode ray tube (CRT), a liquid crystal display (LCD), and a speaker; a storage section 303 including a hard disk and the like; and a communication section 309 including a network interface card such as a LAN (Local Area Network) card or a modem. The communication section 309 performs communication processing via a network such as the Internet. A drive 310 is also connected to the I / O interface 305 as needed. Removable media 311, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, and the like, are installed in the drive 310 as needed so that computer programs read therefrom can be installed into the storage section 308 as needed.

[0147] In particular, according to an embodiment of the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present invention includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 309, and / or installed from the removable medium 311. When the computer program is executed by the central processing unit (CPU) 301, the above-mentioned functions defined in the method of the present application are executed. It should be noted that the computer-readable medium mentioned above in the present application can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium can be, for example, but not limited to, a system, device or component of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above.

[0148] More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection having one or more conductors, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this application, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal may take a variety of forms, including, but not limited to, electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transfer a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code embodied on a computer-readable medium may be transmitted using any suitable medium, including, but not limited to, wireless, wire, optical cable, RF, etc., or any suitable combination thereof.

[0149] Computer program code for performing the operations of the present application may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0150] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, program segment or a part of code, and the module, program segment or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented by a dedicated hardware-based system that performs the specified function or operation, or can be implemented by a combination of dedicated hardware and computer instructions.

[0151] The terms "first", "second", etc. are used to distinguish similar objects, rather than to describe or indicate a particular order or sequence.

[0152] The term "comprise" or any other similar term is intended to cover non-exclusive inclusion such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed or inherent to such process, method, article, or apparatus.

[0153] So far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings.

[0154] The foregoing is merely an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should all be included within the scope of the claims of the present application.

Claims

1. A multi-view extrinsic parameter calibration and plane estimation method, characterized in that: The method comprises: Acquire a target image, where the target image is obtained by sequentially capturing images of a plurality of undetermined planes on which checkerboard grids are placed, using a target camera based on a plurality of preset camera positions; Preprocessing the target image and extracting checkerboard corner points from each preprocessed target image, wherein the preprocessing includes grayscale conversion, filtering, and edge detection preprocessing; Based on the checkerboard corner points in each target image, a target equation group for each target image is jointly established, and a preset optimization algorithm is used to solve the target equation group to obtain the extrinsic parameters of the target camera, wherein the target equation group is constructed based on the mapping relationship between the world coordinate system and the camera coordinate system of the checkerboard corner points in each target image. The preset optimization algorithm solves the minimum value of a preset objective function by iteratively updating algorithm parameters to optimize the extrinsic parameters of the target camera, and the objective function is used to indicate the residual between the predicted pixel coordinates and the actual pixel coordinates; Determine the plane equations of each undetermined plane according to the checkerboard corner points in each target image; The method of establishing target equations for each target image based on the checkerboard corner points in each target image and solving the target equations using a preset optimization algorithm includes: Determine the pixel coordinates and world coordinates of the checkerboard corner points in each target image; Determine the target equation group after the multi-graph combination according to the pixel coordinates and world coordinates of the chessboard corner points, and solve the target equation group to obtain initial extrinsic parameters; The initial extrinsic parameters are optimized using a Levenberg-Marquardt least squares optimization algorithm to obtain the extrinsic parameters of the target camera.

2. The multi-view extrinsic parameter calibration and plane estimation method according to claim 1, characterized in that: The pixel coordinates and world coordinates of the checkerboard corner points in each target image are expressed in homogeneous coordinates as follows: ； The target equation group after the multi-graph combination is: ； Where u and v are the pixel coordinates of the checkerboard corner points in the pixel coordinate system, x, y, and z are the world coordinates of the checkerboard corner points in the world coordinate system, 1 is the component added when the pixel coordinates and world coordinates of the checkerboard corner points are expressed in homogeneous coordinates, and M is the internal parameter of the target camera. is the parameter to be solved, is the extrinsic parameter of the target camera.

3. The multi-view extrinsic parameter calibration and plane estimation method according to claim 1, characterized in that: When jointly calculating the pixel coordinates of the checkerboard corner points in multiple target images and their corresponding world coordinates, the contribution of each target image to the joint result is balanced by setting the corresponding weight of each target image.

4. The multi-view extrinsic parameter calibration and plane estimation method according to claim 1, characterized in that: The method further comprises: Determine the reprojection error of each undetermined plane before and after extrinsic calibration; According to the reprojection error, it is determined whether the extrinsic parameters of the target camera meet a preset accuracy threshold.

5. The multi-view extrinsic parameter calibration and plane estimation method according to claim 1, characterized in that: Checkerboard corner points in any undetermined plane The world coordinates satisfy: ； passing point The normal vector is , then point The plane equation of the undetermined plane is: ； in, , the normal vector It is calculated from the three non-collinear corner points in the chessboard placed on the plane; The homogeneous coordinate form of the normal vector n is , then point The normal vector n satisfies: ； Assume that the checkerboard corner points in another undetermined plane are ,but The world coordinates satisfy: ； in, Represents the position transformation matrix of a set of checkerboard corner points in the world coordinate system; passing point Normal vector satisfy: ； Assumed normal vector The homogeneous coordinate form of is: , then point The plane equation of the undetermined plane is: 。 6. The multi-view extrinsic parameter calibration and plane estimation method according to claim 1, characterized in that: Determining the plane equations of each undetermined plane based on the checkerboard corner points in each target image includes: Determine the coordinates of the checkerboard corner points in the world coordinate system as follows: ； Where, , is the coordinate of each point in the world coordinate system when the chessboard is placed on the kth plane; In the chessboard corner point coordinate collection The fitting plane corresponds to a set of plane normal vectors and offsets to be solved; exist Randomly select three non-collinear points , to construct two direction vectors to solve the plane normal vector and offset corresponding to the fitting plane, and the solution process satisfies: ； Where, ; ; Points The x-coordinate, y-coordinate and z-coordinate of exist In the remaining point set, traverse each point in turn Distance from the fitting plane , and set the distance threshold to determine the Is it less than the distance threshold? is less than the distance threshold, the current point is determined is a point in the plane; After the traversal is completed, The parameters of the fitting plane are optimized by determining the set with the largest number of interior points until the final plane equation is obtained.

7. A multi-view extrinsic parameter calibration and plane estimation system, characterized in that: The system comprises: A data acquisition module is used to acquire a target image, wherein the target image is obtained by sequentially capturing images of a plurality of undetermined planes on which checkerboard grids are placed using a target camera based on a plurality of preset camera positions; A preprocessing module, configured to preprocess the target image and extract checkerboard corner points from each preprocessed target image, wherein the preprocessing includes grayscale conversion, filtering, and edge detection preprocessing; an extrinsic parameter solving module, configured to establish a target equation group for each target image based on the checkerboard corner points in each target image, and solve the target equation group using a preset optimization algorithm to obtain the extrinsic parameters of the target camera, wherein the target equation group is constructed based on the mapping relationship between the world coordinate system and the camera coordinate system of the checkerboard corner points in each target image, and the preset optimization algorithm solves the minimum value of a preset objective function by iteratively updating algorithm parameters to optimize the extrinsic parameters of the target camera, wherein the objective function is used to indicate the residual between the predicted pixel coordinates and the actual pixel coordinates; A plane estimation module is used to determine the plane equations of each undetermined plane based on the checkerboard corner points in each target image; The method of establishing target equations for each target image based on the checkerboard corner points in each target image and solving the target equations using a preset optimization algorithm includes: Determine the pixel coordinates and world coordinates of the checkerboard corner points in each target image; Determine the target equation group after the multi-graph combination according to the pixel coordinates and world coordinates of the chessboard corner points, and solve the target equation group to obtain initial extrinsic parameters; The initial extrinsic parameters are optimized using a Levenberg-Marquardt least squares optimization algorithm to obtain the extrinsic parameters of the target camera.

8. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to at least one of the processors; wherein, The memory stores instructions that can be executed by the processor, and the instructions are used to be executed by the processor to implement the multi-view extrinsic parameter calibration and plane estimation method described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to be executed by the computer to implement the multi-view extrinsic parameter calibration and plane estimation method according to any one of claims 1 to 6.

Images