Camera calibration method and device and target positioning method and device

By establishing distortion-free imaging model and nonlinear fitting algorithm optimization parameters, the problems of complex camera calibration methods and low accuracy are solved, and high-precision camera calibration in complex scenarios are achieved.

CN118967822AActive Publication Date: 2024-11-15RICHFIT INFORMATION TECH +1

Patent Information

Application Number
CN202410869148.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-06-28
Publication Date
2024-11-15
Estimated Expiration
2044-06-28

AI Technical Summary

Technical Problem

The existing camera calibration methods are complex in operation and low in positioning accuracy, and it is difficult to achieve high-precision calibration in complex scenarios.

Method used

By obtaining the environmental images generated by the camera to be calibrated, a distortion-free image model is established, the internal parameter matrix and external parameter matrix are calculated, and the distortion parameters are optimized using a nonlinear fitting algorithm to complete the camera calibration.

Benefits of technology

It simplifies the camera calibration process, improves calibration accuracy, is suitable for monitoring complex scenes, and ensures the accuracy and efficiency of camera calibration.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118967822A_ABST
    Figure CN118967822A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of camera calibration, and provides a camera calibration method and device and a target positioning method and device. The camera calibration method comprises the following steps: acquiring a plurality of environment images generated by a to-be-calibrated camera; a distortionless imaging model corresponding to the environment image is established; calculating an internal parameter matrix and an external parameter matrix of the distortionless imaging model; establishing a distortion fitting model of the environment image; determining distortion parameters corresponding to the distortion fitting model for the environment images before and after distortion by using a nonlinear fitting algorithm; taking the distortion parameter, the internal parameter matrix and the external parameter matrix as decision variables, and establishing a target function of the distortion fitting model under a regular constraint condition; optimizing the target function by using a nonlinear optimization algorithm so as to optimize the distortion parameter, the internal parameter matrix and the external parameter matrix; and completing camera calibration according to the optimized internal parameter matrix, the optimized external parameter matrix and the distortion parameter. According to the embodiment of the invention, the camera calibration efficiency and precision can be improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of camera calibration technology, and in particular to a camera calibration method and device, and a target positioning method and device. Background Art

[0002] Intelligent video surveillance technology uses high-definition cameras and advanced computer vision technology to monitor and analyze visual data of a specific area in real time, thereby automatically detecting and identifying various events and objects, such as people, vehicles, objects, and even behavioral patterns, to provide more efficient monitoring and security management.

[0003] Camera calibration is an important prerequisite for realizing intelligent video surveillance. The accuracy of the calibration results and the stability of the algorithm directly affect the accuracy of the results produced by the camera. The existing camera calibration method requires the use of a calibration object with known size. By establishing the correspondence between the points on the calibration object with known coordinates and their image points, a certain algorithm is used to obtain the internal and external parameters of the camera model. However, the calibration object is always required during the calibration process, and the production accuracy of the calibration object will affect the calibration results. In addition, the calibration object cannot be placed in some scenes, resulting in the existing camera calibration method having the problems of complex camera calibration task operation and low positioning accuracy. Therefore, a camera calibration method is urgently needed to simplify the camera calibration process, improve the camera calibration accuracy, and be applicable to complex scene monitoring such as refining scenes. Summary of the invention

[0004] In view of the fact that the current camera calibration method has the problems of complex camera calibration task operation and low positioning accuracy, this solution is proposed to overcome the above problems or at least partially solve the above problems.

[0005] On the one hand, an object of some embodiments of this specification is to provide a camera calibration method, the method comprising:

[0006] Obtain multiple environment images generated by the camera to be calibrated;

[0007] Establishing a distortion-free imaging model according to a perspective imaging process of a checkerboard calibration plate in the environmental image;

[0008] Calculating the intrinsic parameter matrix and the extrinsic parameter matrix of the distortion-free imaging model;

[0009] Establishing a distortion fitting model of the environmental image;

[0010] For the environmental image before and after the distortion, a nonlinear fitting algorithm is used to determine the distortion parameters corresponding to the distortion fitting model;

[0011] Taking the distortion parameters, the internal parameter matrix and the external parameter matrix as decision variables, an objective function of the distortion fitting model under regularity constraints is established;

[0012] Optimizing the objective function using a nonlinear optimization algorithm to optimize the distortion parameters, the intrinsic parameter matrix and the extrinsic parameter matrix;

[0013] The camera calibration is completed based on the optimized intrinsic parameter matrix, extrinsic parameter matrix and distortion parameters.

[0014] Furthermore, a distortion-free imaging model is established according to the perspective imaging process of the checkerboard calibration plate in the environmental image, including:

[0015] According to two sets of coordinate information of the checkerboard calibration plate in the world coordinate system and the pixel coordinate system and the perspective imaging process, determining the perspective imaging relationship of the checkerboard calibration plate from the world coordinate system to the camera coordinate system and the pixel coordinate system in sequence;

[0016] Perform a product operation on a mapping relationship of the checkerboard calibration plate from the world coordinate system to the camera coordinate system and a mapping relationship of the checkerboard calibration plate from the camera coordinate system to the pixel coordinate system to obtain a product operation matrix;

[0017] According to the product operation matrix, the checkerboard calibration plate is mapped from the world coordinate system to the pixel coordinate system to establish the distortion-free imaging model.

[0018] Furthermore, before calculating the intrinsic parameter matrix and the extrinsic parameter matrix of the distortion-free imaging model, the method further includes:

[0019] Acquire a color calibration plate image, pixel coordinates of preset calibration points in the color calibration plate image, and three-dimensional coordinates of the preset calibration points;

[0020] Inputting the color calibration plate image into a pre-trained neural network model to identify and obtain pixel coordinates of a preset area in the color calibration plate image;

[0021] Establishing a coordinate mapping model between the pixel coordinates of the preset calibration points and the three-dimensional coordinates of the preset calibration points;

[0022] Inputting the pixel coordinates of the preset area into the coordinate mapping model to obtain the three-dimensional coordinates of the preset area;

[0023] According to the three-dimensional coordinates of the preset area, coordinate information of a plurality of target points in the checkerboard calibration plate having the same position as the color calibration plate in the world coordinate system is determined.

[0024] Furthermore, calculating the intrinsic parameter matrix and the extrinsic parameter matrix of the distortion-free imaging model includes:

[0025] Inputting coordinate information of a plurality of target points in the chessboard calibration plate in the world coordinate system and coordinate information of the target points in the pixel coordinate system into the undistorted imaging model to obtain a homography matrix;

[0026] Determining the constraint equation corresponding to the symmetric matrix of the intrinsic parameter matrix according to the coordinate information of the target points of the checkerboard calibration plate in the plurality of environmental images in the pixel coordinate system;

[0027] Based on the constraint equation, the symmetric matrix is ​​calculated using a fitting algorithm;

[0028] Determine an internal parameter matrix corresponding to the symmetric matrix;

[0029] The product of the inverse matrix of the intrinsic parameter matrix and the homography matrix is ​​used as the extrinsic parameter matrix.

[0030] Further, using a nonlinear fitting algorithm, the distortion parameters corresponding to the distortion fitting model are determined, including:

[0031] Inputting coordinate information of a plurality of target points in the checkerboard calibration plate in the pixel coordinate system into the distortion fitting model to obtain a plurality of constraint equations constructed by distortion parameters;

[0032] A plurality of constraint equations constructed by distortion parameters are fitted using a nonlinear fitting algorithm, and the distortion parameters are obtained according to the fitting results.

[0033] Furthermore, the distortion parameters, the internal parameter matrix and the external parameter matrix are used as decision variables to establish the objective function of the distortion fitting model under regularity constraints, including:

[0034] Taking the distortion parameters, the internal parameter matrix and the external parameter matrix as decision variables, determining the difference between the environment image before and after the distortion under the current decision variables and the regularization terms corresponding to the distortion parameters, the internal parameter matrix and the external parameter matrix;

[0035] The objective function is established by performing a weighted summation on the difference degree and the regularization term.

[0036] Furthermore, the objective function is optimized using a nonlinear optimization algorithm to optimize the distortion parameters, the intrinsic parameter matrix and the extrinsic parameter matrix, including:

[0037] Take the current decision variable as the current iteration point;

[0038] Constructing a similar function to the target function in the neighborhood of the current iteration point;

[0039] Calculate the minimum value corresponding to the similarity function in the neighborhood, and use the minimum point corresponding to the minimum value as the next iteration point;

[0040] Calculating the similarity between the target function and the similarity function according to the current iteration point and the next iteration point;

[0041] If the similarity is greater than a first threshold, taking the next iteration point as the current iteration point;

[0042] If the similarity is not greater than the first threshold, determining an iteration step length according to the similarity, and iterating the current iteration point using the iteration step length, so as to use the iteration result as the current iteration point;

[0043] Repeat the above steps of updating the current iteration point until the objective function converges.

[0044] Based on the same inventive concept, some embodiments of this specification also provide a target positioning method, the method comprising:

[0045] Receiving an image to be positioned; the image to be positioned is generated by a camera calibrated using the method described in any of the aforementioned embodiments;

[0046] The image to be positioned is input into a pre-trained target positioning model to obtain a positioning result.

[0047] On the other hand, some embodiments of this specification further provide a camera calibration device, the device comprising:

[0048] An acquisition module, used to acquire multiple environment images generated by the camera to be calibrated;

[0049] An imaging modeling module, used to establish a distortion-free imaging model according to the perspective imaging process of the chessboard calibration plate in the environmental image;

[0050] A parameter calculation module, used to calculate the internal parameter matrix and the external parameter matrix of the distortion-free imaging model;

[0051] A distortion modeling module, used to establish a distortion fitting model of the environment image;

[0052] A distortion fitting module, used to determine the distortion parameters corresponding to the distortion fitting model using a nonlinear fitting algorithm for the environmental image before and after distortion;

[0053] An objective function establishment module is used to establish an objective function of the distortion fitting model under regularity constraints by taking the distortion parameters, the internal parameter matrix and the external parameter matrix as decision variables;

[0054] An optimization module, used for optimizing the objective function by using a nonlinear optimization algorithm to optimize the distortion parameters, the intrinsic parameter matrix and the extrinsic parameter matrix;

[0055] The calibration module is used to complete the camera calibration according to the optimized intrinsic parameter matrix, extrinsic parameter matrix and distortion parameters.

[0056] On the other hand, based on the same inventive concept, some embodiments of this specification further provide a target positioning device, the device comprising:

[0057] A receiving module, used for receiving an image to be positioned; the image to be positioned is generated by a camera calibrated by the method described in any of the above embodiments;

[0058] The positioning module is used to input the image to be positioned into a pre-trained target positioning model to obtain a positioning result.

[0059] On the other hand, some embodiments of the present specification further provide a computer device, including a memory, a processor, and a computer program stored in the memory, wherein when the computer program is executed by the processor, the instructions of the above method are executed.

[0060] On the other hand, some embodiments of the present specification further provide a computer storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor of a computer device, the instructions of the above method are executed.

[0061] On the other hand, some embodiments of the present specification further provide a computer program product, wherein the computer program product includes a computer program, and when the computer program is executed by a processor of a computer device, the instructions of the above method are executed.

[0062] One or more technical solutions provided by some embodiments of this specification have at least the following technical effects:

[0063] The embodiments of the present specification first automatically obtain multiple environmental images generated by the camera to be calibrated, and then establish a distortion-free imaging model based on the perspective imaging process of the checkerboard calibration plate in the environmental image to determine the intrinsic parameter matrix corresponding to the camera and the extrinsic parameter matrix corresponding to the environmental image under the distortion-free condition, thereby establishing a distortion fitting model of the environmental image, and using a nonlinear fitting algorithm to determine the corresponding distortion parameters. However, since the obtained distortion parameters, intrinsic parameter matrix and extrinsic parameter matrix are obtained by fitting through an approximate method, an objective function of the distortion fitting model under regularization constraints is established, and a nonlinear optimization algorithm is used to optimize the objective function, thereby optimizing the distortion parameters, intrinsic parameter matrix and extrinsic parameter matrix, so as to ensure that the camera calibration accuracy is improved under the premise of camera calibration efficiency.

[0064] The above description is only an overview of the technical solutions of some embodiments of this specification. In order to more clearly understand the technical means of some embodiments of this specification, they can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of some embodiments of this specification more obvious and easy to understand, the specific implementation methods of some embodiments of this specification are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] In order to more clearly illustrate some embodiments of this specification or technical solutions in the prior art, the following briefly introduces the drawings required for the embodiments or prior art descriptions. Obviously, the drawings described below are only some embodiments recorded in this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor. In the drawings:

[0066] Figure 1 A schematic diagram of an implementation system of a camera calibration method in some embodiments of this specification is shown;

[0067] Figure 2 A flowchart of a camera calibration method in some embodiments of this specification is shown;

[0068] Figure 3 A schematic diagram of the steps of establishing a distortion-free imaging model in some embodiments of this specification;

[0069] Figure 4 A schematic diagram of a process of distortion-free imaging in some embodiments of this specification;

[0070] Figure 5 A schematic diagram of the steps of determining the coordinate information of multiple target points in a chessboard calibration plate in a world coordinate system in some embodiments of this specification;

[0071] Figure 6 A schematic diagram of the steps of calculating the internal parameter matrix and the external parameter matrix in some embodiments of this specification;

[0072] Figure 7 A schematic diagram of the steps of fitting to obtain distortion parameters in some embodiments of this specification;

[0073] Figure 8 A schematic diagram of the steps of establishing an objective function in some embodiments of this specification;

[0074] Fig. 9 A schematic diagram of the steps of optimizing distortion parameters, intrinsic parameter matrices and extrinsic parameter matrices in some embodiments of this specification;

[0075] Fig.10 A flow chart of a target positioning method in some embodiments of this specification is shown;

[0076] Fig.11 This is a schematic diagram of the structure of a camera calibration device in some embodiments of this specification;

[0077] Fig.12 This is a schematic diagram of the structure of a target positioning device in some embodiments of this specification;

[0078] Fig.13 This is a schematic diagram of the computer device structure provided in some embodiments of this specification.

[0079] [Description of Reference Numerals]

[0080] 101. Terminal;

[0081] 102. Server;

[0082] 1101. Obtain module;

[0083] 1102. Imaging modeling module;

[0084] 1103. parameter calculation module;

[0085] 1104. Distortion modeling module;

[0086] 1105. distortion fitting module;

[0087] 1106. Objective function establishment module;

[0088] 1107. Optimization module;

[0089] 1108, calibration module;

[0090] 1201, receiving module;

[0091] 1202, positioning module;

[0092] 1302. Computer equipment;

[0093] 1304, processor;

[0094] 1306. Memory;

[0095] 1308, driving mechanism;

[0096] 1310, input / output interface;

[0097] 1312. Input devices;

[0098] 1314. Output device;

[0099] 1316. Presentation equipment;

[0100] 1318. Graphical user interface;

[0101] 1320, network interface;

[0102] 1322, communication link;

[0103] 1324. Communication bus. DETAILED DESCRIPTION

[0104] In order to enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in some embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on some embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this specification.

[0105] It should be noted that the terms "first", "second", etc. in the specification and claims of this document and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of this document described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, device, product or equipment that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or equipment. It should be noted that the acquisition, storage, use, processing, etc. of data in the technical solution of this application comply with the relevant provisions of relevant laws and regulations.

[0106] like Figure 1 The diagram is a schematic diagram of a system for implementing a camera calibration method according to an embodiment of the present invention, which may include: a terminal 101 and a server 102, wherein the terminal 101 stores multiple images taken by a camera to be calibrated, and the terminal 101 and the server 102 communicate via a network, which may include a local area network (LAN), a wide area network (WAN), the Internet, or a combination thereof, and is connected to a website, a user device (such as a computing device), and a backend system. A staff member may send a camera calibration request to the server 102 via the terminal 101, and after receiving the camera calibration request, the server 102 calls the data in the database for calculation and processing to obtain a camera calibration result, and sends the camera calibration result to the terminal 101, so that the staff member can process the business according to the camera calibration result.

[0107] In the embodiments of this specification, the server 102 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN, Content Delivery Network), and big data and artificial intelligence platforms.

[0108] In an optional embodiment, the terminal 101 may include but is not limited to electronic devices such as self-service terminal devices, desktop computers, tablet computers, laptop computers, smart wearable devices, etc., which can be connected to the camera to be calibrated. Optionally, the operating system running on the electronic device may include but is not limited to Android system, IOS system, Linux, Windows, etc. Of course, the terminal 101 is not limited to the above-mentioned electronic devices with a certain entity, and it can also be software running in the above-mentioned electronic devices.

[0109] In addition, it should be noted that Figure 1 What is shown is only an application environment provided by the present disclosure. In actual application, multiple terminals 101 may be included, and this specification does not limit this.

[0110] Figure 2 This is a flowchart of a camera calibration method provided by an embodiment of the present invention. This specification provides method operation steps as described in the embodiment or flowchart, but may include more or fewer operation steps based on conventional or non-creative labor. The order of steps listed in the embodiment is only one way of executing the steps among many orders, and does not represent the only order of execution. When the actual system or device product is executed, it can be executed in the order of the method shown in the embodiment or the accompanying drawings or in parallel. Specifically, Figure 2 As shown, applied to the above-mentioned server side, the method may include:

[0111] S201: Acquire multiple environment images generated by the camera to be calibrated;

[0112] S202: establishing a distortion-free imaging model according to the perspective imaging process of the checkerboard calibration plate in the environment image;

[0113] S203: Calculating the internal parameter matrix and the external parameter matrix of the distortion-free imaging model;

[0114] S204: Establishing a distortion fitting model of the environment image;

[0115] S205: Determine distortion parameters corresponding to the distortion fitting model using a nonlinear fitting algorithm for the environmental image before and after distortion;

[0116] S206: Using the distortion parameters, the internal parameter matrix and the external parameter matrix as decision variables, and establishing an objective function of the distortion fitting model under regularity constraints;

[0117] S207: optimizing the objective function using a nonlinear optimization algorithm to optimize the distortion parameters, the intrinsic parameter matrix and the extrinsic parameter matrix;

[0118] S208: Complete camera calibration according to the optimized intrinsic parameter matrix, extrinsic parameter matrix and distortion parameters.

[0119] The embodiments of the present specification first automatically obtain multiple environmental images generated by the camera to be calibrated, and then establish a distortion-free imaging model based on the perspective imaging process of the checkerboard calibration plate in the environmental image to determine the intrinsic parameter matrix corresponding to the camera and the extrinsic parameter matrix corresponding to the environmental image under the distortion-free condition, thereby establishing a distortion fitting model of the environmental image, and using a nonlinear fitting algorithm to determine the corresponding distortion parameters. However, since the obtained distortion parameters, intrinsic parameter matrix and extrinsic parameter matrix are obtained by fitting through an approximate method, an objective function of the distortion fitting model under regularization constraints is established, and a nonlinear optimization algorithm is used to optimize the objective function, thereby optimizing the distortion parameters, intrinsic parameter matrix and extrinsic parameter matrix, so as to ensure that the camera calibration accuracy is improved under the premise of camera calibration efficiency.

[0120] It can be understood that, in some embodiments, a spatial rectangular coordinate system is established with the earth plane as the XY plane; the origin of the real space rectangular coordinate system is determined, and the longitude and latitude of the origin are measured using a GPS locator, a target reference point is selected, and the longitude and latitude of the target reference point are measured using a GPS locator, the due east direction is determined using a GPS locator, and the deflection angle between the due east direction and the positive direction of the X-axis is measured, for one or more cameras to be calibrated in the real space rectangular coordinate system, a checkerboard calibration plate is placed in the real space, and the world coordinate system is fixed on the checkerboard, so that W=0 in the physical coordinates (U, V, W) of any point on the checkerboard, and the camera to be calibrated is used to shoot it at different angles to obtain multiple environmental images generated by the camera to be calibrated, thereby establishing a distortion-free imaging model and a distortion fitting model based on the imaging process, and establishing a corresponding objective function for optimizing the parameters of the distortion fitting model, which is an approximate fitting model, thereby improving the accuracy of the distortion parameters, the intrinsic parameter matrix and the extrinsic parameter matrix, so as to improve the accuracy of the camera calibration.

[0121] It should be noted that, in some embodiments, due to the complex perspective process of the fisheye camera and its application limitations, the camera to be calibrated is an ordinary camera, not a fisheye camera.

[0122] Refer to the attached Figure 3In some embodiments, establishing a distortion-free imaging model according to the perspective imaging process of the checkerboard calibration plate in the environment image may include:

[0123] S301: determining a perspective imaging relationship of the checkerboard calibration plate from the world coordinate system to the camera coordinate system and the pixel coordinate system in sequence according to two sets of coordinate information of the checkerboard calibration plate in the world coordinate system and the pixel coordinate system and a perspective imaging process;

[0124] S302: performing a product operation on a mapping relationship of the checkerboard calibration plate from the world coordinate system to the camera coordinate system and a mapping relationship of the checkerboard calibration plate from the camera coordinate system to the pixel coordinate system to obtain a product operation matrix;

[0125] S303: Mapping the checkerboard calibration plate from the world coordinate system to the pixel coordinate system according to the product operation matrix to establish the distortion-free imaging model.

[0126] It can be understood that in some embodiments, referring to the attached Figure 4 , the camera imaging system contains four coordinate systems: world coordinate system, camera coordinate system, image coordinate system, and pixel coordinate system. The world coordinate system is the coordinate system in the real world. The environmental objects are first radially mapped from the world coordinate system to the camera coordinate system through rigid body transformation, and then mapped from the camera coordinate system to the image coordinate system through perspective projection, and then mapped from the image coordinate system to the pixel coordinate system through affine transformation. Among them, the product of the affine transformation matrix and the perspective projection matrix is ​​the intrinsic parameter matrix, and the rigid body transformation matrix is ​​the extrinsic parameter matrix. Therefore, when establishing a distortion-free imaging model, it is only necessary to determine the mapping relationship of the checkerboard calibration plate from the world coordinate system to the camera coordinate system, and the mapping relationship of the checkerboard calibration plate from the camera coordinate system to the pixel coordinate system. There is no need to consider the image coordinate system, so that the distortion-free imaging model can be established quickly and accurately.

[0127] Refer to the attached Figure 5 In some embodiments, before calculating the intrinsic parameter matrix and the extrinsic parameter matrix of the distortion-free imaging model, the following may be further included:

[0128] S501: Acquire a color calibration plate image, pixel coordinates of preset calibration points in the color calibration plate image, and three-dimensional coordinates of the preset calibration points;

[0129] S502: Inputting the color calibration plate image into a pre-trained neural network model to identify and obtain pixel coordinates of a preset area in the color calibration plate image;

[0130] S503: Establishing a coordinate mapping model between the pixel coordinates of the preset calibration point and the three-dimensional coordinates of the preset calibration point;

[0131] S504: Inputting the pixel coordinates of the preset area into the coordinate mapping model to obtain the three-dimensional coordinates of the preset area;

[0132] S505: Determine coordinate information of a plurality of target points in the checkerboard calibration plate at the same position as the color calibration plate in the world coordinate system according to the three-dimensional coordinates of the preset area.

[0133] It can be understood that, in some embodiments, after establishing the distortion-free imaging model, the coordinate information of the target point of the checkerboard calibration plate in the environment image in the world coordinate system and the pixel coordinate system can be substituted into the distortion-free imaging model to solve the intrinsic parameter matrix and the extrinsic parameter matrix. The target point of the checkerboard calibration plate can be a corner point or other point. However, manually measuring the coordinate information of the target point of the checkerboard calibration plate in the world coordinate system has the problems of poor accuracy, long time consumption and low efficiency. Therefore, a pre-trained neural network model can be used, such as the YOLOV5 target detection network. The neural network model is used to identify the pixel coordinates of a preset position in the color calibration plate image. Specifically, the color calibration plate can be a calibration plate of different colors, and the calibration plate can include a checkerboard calibration plate, a circular grid calibration plate, and a color calibration plate. As well as other types of calibration plates, the preset area may be a designated position area, such as a designated area in the upper right corner. The pixel coordinates of the preset calibration point in the color calibration plate image and the three-dimensional coordinates of the preset calibration point are a small amount of coordinate information obtained by pre-collecting and measuring to establish a coordinate mapping model between the pixel coordinates of the preset calibration point and the three-dimensional coordinates of the preset calibration point. By using the coordinate mapping model, the three-dimensional coordinates of the preset area in the color calibration plate image (i.e., the three-dimensional coordinates in the real world) can be quickly and accurately obtained. Since the checkerboard calibration plate and the color calibration plate have the same position, several points can be selected from the preset area in the color calibration plate image as target points in the checkerboard calibration plate, thereby quickly and accurately determining the coordinate information of multiple target points in the checkerboard calibration plate in the world coordinate system.

[0134] Refer to the attached Figure 6 In some embodiments, calculating the intrinsic parameter matrix and the extrinsic parameter matrix of the distortion-free imaging model may include:

[0135] S601: inputting coordinate information of multiple target points in the chessboard calibration plate in the world coordinate system and coordinate information of the target points in the pixel coordinate system into the undistorted imaging model to obtain a homography matrix;

[0136] S602: determining the constraint equation corresponding to the symmetric matrix of the intrinsic parameter matrix according to the coordinate information of the target point of the checkerboard calibration plate in the plurality of environment images in the pixel coordinate system;

[0137] S603: Based on the constraint equation, the symmetric matrix is ​​calculated using a fitting algorithm;

[0138] S604: Determine an internal parameter matrix corresponding to the symmetric matrix;

[0139] S605: Taking the product of the inverse matrix of the intrinsic parameter matrix and the homography matrix as the extrinsic parameter matrix.

[0140] It can be understood that, in some embodiments, according to the camera imaging principle, when calculating the intrinsic parameter matrix and the extrinsic parameter matrix of the distortion-free imaging model, it is first necessary to solve the product of the intrinsic parameter matrix and the extrinsic parameter matrix, that is, the homography matrix, and simplify the distortion-free imaging model to obtain:

[0141] Z·P=A·W·R=H·R

[0142] Among them, Z is the scale factor, is a constant, P is the coordinate information of the target point in the pixel coordinate system, H is the homography matrix, R is the coordinate information of the target point in the world coordinate system, A is the intrinsic parameter matrix, and W is the extrinsic parameter matrix.

[0143] Furthermore, in some embodiments, the scale factor can be eliminated and the homography matrix can be obtained through the coordinate information of at least 4 target points in the pixel coordinate system and the coordinate information in the world coordinate system. The reason why at least 4 target points are needed is that in the calculation process, each target point is represented by a three-dimensional vector, which can provide 2 constraint equations, and H after simplification is a homogeneous matrix containing 8 independent unknown elements, so the number of target points is at least 4.

[0144] Furthermore, in some embodiments, it is known that A·W=H, and it is necessary to solve the internal parameter matrix A, where W=(R1, R2, T), T is the translation vector, R=(R1, R2), R is the rotation matrix, R1 and R2 are two columns of the rotation matrix, and R1 and R2 are orthogonal. It can be seen that R1 T R2=0,R1 T R1=R2 T R2 = 1, and H = A (R1, R2, T), the constraint equation corresponding to the symmetric matrix of the internal parameter matrix can be obtained by combining them. Specifically, the symmetric matrix of the internal parameter matrix is ​​recorded as B = A -T A -1 , the constraint equations include: H1 T B H2=0,H1 T ·B·H1=H2 T·B·H2=1, where H1 and H2 are the first two column vectors of H respectively. Since the camera intrinsic parameters are parameters describing the inherent properties of the camera itself, including parameters such as focal length and pixel spacing, they are usually represented by an intrinsic parameter matrix. The camera extrinsic parameters are parameters describing the position and orientation of the camera in a three-dimensional scene, usually represented by a rotation matrix and a translation vector. For the same camera to be calibrated, its intrinsic parameter matrix depends on its internal parameters. No matter how the positional relationship between the checkerboard calibration plate and the camera to be calibrated changes, the intrinsic parameter matrix remains unchanged, while the extrinsic parameter matrix reflects the positional relationship between the checkerboard calibration plate and the camera to be calibrated. For different pictures, if the positional relationship between the checkerboard calibration plate and the camera to be calibrated has changed, the extrinsic parameter matrix corresponding to each picture is different. Therefore, when using the fitting algorithm to solve the symmetric matrix under the constraint equation, the coordinate information of the target points corresponding to the world coordinate system and the pixel coordinate system of the number of different environment images greater than the preset threshold is required. Usually, the preset threshold exceeds 10 to ensure sufficiently accurate fitting accuracy. After solving the symmetric matrix B, B=A -T A -1 Perform operations to obtain the internal parameter matrix A, and since W = A -1 H, so the external parameter matrix W can be calculated, so as to quickly and accurately obtain the internal parameter matrix and external parameter matrix of the distortion-free imaging model.

[0145] Refer to the attached Figure 7 In some embodiments, determining the distortion parameters corresponding to the distortion fitting model using a nonlinear fitting algorithm may include:

[0146] S701: inputting coordinate information of a plurality of target points in the checkerboard calibration plate in the pixel coordinate system into the distortion fitting model to obtain a plurality of constraint equations constructed by distortion parameters;

[0147] S702: Fitting a plurality of constraint equations constructed by distortion parameters using a nonlinear fitting algorithm, and obtaining the distortion parameters according to the fitting results.

[0148] It can be understood that, in some embodiments, the distortion fitting model maps the undistorted image coordinates to the distorted image coordinates through the distortion parameters, the distortion parameters can be regarded as unknowns, and a nonlinear fitting algorithm is used to fit multiple constraint equations constructed by the distortion parameters, so as to obtain the distortion parameters according to the fitting results. However, the conventional distortion fitting model only considers the radial distortion with a greater influence, and there is a defect that the distortion fitting model has a low modeling accuracy, and there is room for further optimization.

[0149] Refer to the attached Figure 8 In some embodiments, the distortion parameters, the internal parameter matrix and the external parameter matrix are used as decision variables to establish the objective function of the distortion fitting model under the regularity constraint condition, which may include:

[0150] S801: Taking the distortion parameters, the internal parameter matrix and the external parameter matrix as decision variables, determining the difference between the environment image before and after the distortion under the current decision variables and the regularization terms corresponding to the distortion parameters, the internal parameter matrix and the external parameter matrix;

[0151] S802: Perform weighted summation on the difference degree and the regularization term to establish the objective function.

[0152] It can be understood that, in some embodiments, the distortion fitting model under the multivariate distortion parameter can further improve the modeling accuracy of the distortion fitting model, which can be expressed as:

[0153]

[0154] Among them, (x d ,y d ) is the image coordinate after distortion, (x, y) is the image coordinate before distortion, r is the distance from the image pixel to the image center, k1, k2, k3, p1, p2, s1, s2, s3, s4 are distortion parameters.

[0155] Furthermore, in some embodiments, the objective function can be expressed as:

[0156] L = ω1 × argmin w MSE(F(x d ,y d ), (x, y))+ω2×||w||

[0157] Among them, L represents the objective function, ω1 represents the first weight, argmin w Indicates that MSE(F(x d ,y d ),(x,y)) reaches the minimum value, MSE represents the mean square error, which is used to measure the difference between the environment image before and after the distortion, F represents the distortion fitting model under the multivariate distortion parameter, ω2 represents the second weight, ||w|| represents the L2 norm, (x d ,y d ) is the image coordinate after distortion, and (x, y) is the image coordinate before distortion.

[0158] Refer to the attached Fig. 9 In some embodiments, optimizing the objective function using a nonlinear optimization algorithm to optimize the distortion parameters, the intrinsic parameter matrix and the extrinsic parameter matrix may include:

[0159] S901: taking the current decision variable as the current iteration point;

[0160] S902: constructing a similar function to the target function in the neighborhood of the current iteration point;

[0161] S903: Calculate the minimum value corresponding to the similarity function in the neighborhood, and use the minimum point corresponding to the minimum value as the next iteration point;

[0162] S904: Calculating the similarity between the target function and the similarity function according to the current iteration point and the next iteration point;

[0163] S905: If the similarity is greater than a first threshold, taking the next iteration point as the current iteration point;

[0164] S906: If the similarity is not greater than the first threshold, determining an iteration step according to the similarity, and iterating the current iteration point using the iteration step to use the iteration result as the current iteration point;

[0165] S907: Repeat the above steps of updating the current iteration point until the objective function converges.

[0166] It can be understood that, in some embodiments, after the objective function is established, the distortion fitting model established in advance performs a distortion fitting operation, and the initialized distortion parameters, internal parameter matrix and external parameter matrix are obtained. However, the distortion fitting operation has a certain distortion fitting error. In order to further ensure the accuracy of the distortion parameters, the internal parameter matrix and the external parameter matrix, while taking into account the optimization speed, the optimization process can be started around the initial value of the current decision variable. Specifically, the current decision variable is first used as the current iteration point. At the same time, in order to avoid local convergence and reduce the sensitivity to the initial value, a similar function to the objective function in the neighborhood of the current iteration point is constructed. It should be noted that the similarity function is composed of simple functions. The simple function does not refer to a simple function, but corresponds to the concept in the theory of real variables. Then, the minimum value corresponding to the similarity function in the neighborhood is calculated, and the minimum point corresponding to the minimum value is used as the next iteration point to achieve update iteration. In order to evaluate the effectiveness of the iteration step, the similarity between the objective function and the similarity function is calculated using the following formula:

[0167]

[0168] Among them, σ i is the similarity degree at the i-th iteration, ||L i || 2 -||L(x i +d i )|| 2 is the actual reduction of the objective function at the i-th iteration, ||L i || 2 -||Li +J i d i || 2 is the predicted reduction of the objective function at the i-th iteration, L i is the objective function value before the i-th iteration, L(x i +d i ) is the objective function value after the i-th iteration, J i d i is the predicted iterative change at the current iteration step size at the i-th iteration, d i is the iteration step size at the i-th iteration.

[0169] Furthermore, in some embodiments, if the degree of similarity is greater than a first threshold, it means that the degree of approximation between the similarity function and the target function is high, and the current iteration is valid; otherwise, it means that the degree of approximation between the similarity function and the target function is low, and the current iteration is not accepted. The iteration step is adjusted according to the degree of similarity; if the degree of similarity is less than a second threshold, the iteration step is increased; if the degree of similarity is greater than the second threshold (and less than the first threshold), the iteration step is reduced to obtain an updated iteration step; thereafter, the current iteration point is iterated using the updated iteration step, and the steps of updating the current iteration point are repeated until the target function converges, thereby quickly and accurately finding the optimal solution in a credible neighborhood to obtain the optimized distortion parameters, intrinsic parameter matrix, and extrinsic parameter matrix.

[0170] Based on the same inventive concept, some embodiments of this specification also provide a target positioning method, referring to the attached Fig.10 , the method comprising:

[0171] S1001: receiving an image to be positioned; the image to be positioned is generated by a camera calibrated using the method described in any of the above embodiments;

[0172] S1002: Input the image to be positioned into a pre-trained target positioning model to obtain a positioning result.

[0173] It should be noted that, although the operations of the method of the present invention are described in a specific order in the above embodiments and the accompanying drawings, this does not require or imply that the operations must be performed in the specific order, or that all the operations shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps.

[0174] Corresponding to the above-mentioned camera calibration method, some embodiments of this specification further provide a camera calibration device, referring to Fig.11 As shown, in some embodiments, the apparatus may include:

[0175] An acquisition module 1101 is used to acquire multiple environment images generated by the camera to be calibrated;

[0176] An imaging modeling module 1102, configured to establish a distortion-free imaging model according to a perspective imaging process of a checkerboard calibration plate in the environment image;

[0177] A parameter calculation module 1103 is used to calculate the internal parameter matrix and the external parameter matrix of the distortion-free imaging model;

[0178] A distortion modeling module 1104 is used to establish a distortion fitting model of the environment image;

[0179] The distortion fitting module 1105 is used to determine the distortion parameters corresponding to the distortion fitting model using a nonlinear fitting algorithm for the environmental image before and after the distortion;

[0180] An objective function establishment module 1106 is used to establish an objective function of the distortion fitting model under regularity constraints by taking the distortion parameters, the internal parameter matrix and the external parameter matrix as decision variables;

[0181] An optimization module 1107, configured to optimize the objective function using a nonlinear optimization algorithm to optimize the distortion parameters, the intrinsic parameter matrix and the extrinsic parameter matrix;

[0182] The calibration module 1108 is used to complete the camera calibration according to the optimized intrinsic parameter matrix, extrinsic parameter matrix and distortion parameters.

[0183] Based on the same inventive concept, corresponding to the above-mentioned target positioning method, some embodiments of this specification further provide a target positioning device, referring to Fig.12 As shown, in some embodiments, the apparatus may include:

[0184] The receiving module 1201 is used to receive an image to be positioned; the image to be positioned is generated by a camera calibrated by the method described in any of the above embodiments;

[0185] The positioning module 1202 is used to input the image to be positioned into a pre-trained target positioning model to obtain a positioning result.

[0186] For the convenience of description, the above device is described in various units according to their functions. Of course, when implementing this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0187] It should be noted that in the embodiments of this specification, the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved are all information and data authorized by the user and fully authorized by all parties.

[0188] It should be noted that the computer program product described in this specification (this application) is a software product that mainly implements the method described in this specification (this application) through a computer program.

[0189] The embodiments of this specification also provide a computer device. Fig.13 As shown, in some embodiments of the present specification, the computer device 1302 may include one or more processors 1304, such as one or more central processing units (CPUs) or graphics processing units (GPUs), and each processing unit may implement one or more hardware threads. The computer device 1302 may also include any memory 1306, which is used to store any kind of information such as code, settings, data, etc. In a specific embodiment, the computer program on the memory 1306 and can be run on the processor 1304, when the computer program is run by the processor 1304, the instructions of the method described in any of the above embodiments may be executed. Non-limitingly, for example, the memory 1306 may include any one or more combinations of the following: any type of RAM, any type of ROM, flash memory device, hard disk, optical disk, etc. More generally, any memory can use any technology to store information. Further, any memory can provide volatile or non-volatile retention of information. Further, any memory can represent a fixed or removable component of the computer device 1302. In one embodiment, when the processor 1304 executes the associated instructions stored in any memory or combination of memories, the computer device 1302 can perform any operation of the associated instructions. The computer device 1302 also includes one or more drive mechanisms 1308 for interacting with any memory, such as a hard disk drive mechanism, an optical disk drive mechanism, etc.

[0190] The computer device 1302 may also include an input / output interface 1310 (I / O) for receiving various inputs (via input devices 1312) and for providing various outputs (via output devices 1314). A specific output mechanism may include a presentation device 1316 and an associated graphical user interface 1318 (GUI). In other embodiments, the input / output interface 1310 (I / O), input device 1312, and output device 1314 may not be included, and the computer device 1302 may be used as a computer device in a network. The computer device 1302 may also include one or more network interfaces 1320 for exchanging data with other devices via one or more communication links 1322. One or more communication buses 1324 couple the components described above together.

[0191] The communication link 1322 may be implemented in any manner, for example, through a local area network, a wide area network (e.g., the Internet), a point-to-point connection, etc., or any combination thereof. The communication link 1322 may include any combination of hardwired links, wireless links, routers, gateway functions, name servers, etc., governed by any protocol or combination of protocols.

[0192] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), computer-readable storage media, and computer program products of some embodiments of the present specification. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processor to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processor generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0193] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processor to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0194] These computer program instructions can also be loaded onto a computer or other programmable data processor so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0195] In a typical configuration, a computer device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0196] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0197] Computer-readable media include permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. Information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computer device. As defined in this specification, computer-readable media does not include temporary computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0198] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems or computer program products. Therefore, the embodiments of this specification may take the form of complete hardware embodiments, complete software embodiments or embodiments combining software and hardware. Moreover, the embodiments of this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0199] The present specification embodiments may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present specification embodiments may also be practiced in distributed computing environments where tasks are performed by remote processors connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0200] It should also be understood that in the embodiments of this specification, the term "and / or" is only a description of the association relationship of the associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0201] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0202] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the embodiments of this specification. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0203] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A camera calibration method, characterized in that: The method comprises: Obtain multiple environment images generated by the camera to be calibrated; Establishing a distortion-free imaging model according to a perspective imaging process of a checkerboard calibration plate in the environmental image; Calculating an internal parameter matrix and an external parameter matrix of the distortion-free imaging model; Establishing a distortion fitting model of the environmental image; For the environmental image before and after distortion, a nonlinear fitting algorithm is used to determine the distortion parameters corresponding to the distortion fitting model; Taking the distortion parameters, the internal parameter matrix and the external parameter matrix as decision variables, an objective function of the distortion fitting model under regularity constraints is established; Optimizing the objective function using a nonlinear optimization algorithm to optimize the distortion parameters, the intrinsic parameter matrix and the extrinsic parameter matrix; The camera calibration is completed based on the optimized intrinsic parameter matrix, extrinsic parameter matrix and distortion parameters.

2. The method according to claim 1, characterized in that Establishing a distortion-free imaging model according to the perspective imaging process of the chessboard calibration plate in the environment image includes: According to two sets of coordinate information of the checkerboard calibration plate in the world coordinate system and the pixel coordinate system and the perspective imaging process, determining the perspective imaging relationship of the checkerboard calibration plate from the world coordinate system to the camera coordinate system and the pixel coordinate system in sequence; Performing a product operation on a mapping relationship of the checkerboard calibration plate from the world coordinate system to the camera coordinate system and a mapping relationship of the checkerboard calibration plate from the camera coordinate system to the pixel coordinate system to obtain a product operation matrix; According to the product operation matrix, the checkerboard calibration plate is mapped from the world coordinate system to the pixel coordinate system to establish the distortion-free imaging model.

3. The method according to claim 2, characterized in that Before calculating the intrinsic parameter matrix and the extrinsic parameter matrix of the distortion-free imaging model, the method further includes: Acquire a color calibration plate image, pixel coordinates of preset calibration points in the color calibration plate image, and three-dimensional coordinates of the preset calibration points; Inputting the color calibration plate image into a pre-trained neural network model to identify and obtain pixel coordinates of a preset area in the color calibration plate image; Establishing a coordinate mapping model between the pixel coordinates of the preset calibration points and the three-dimensional coordinates of the preset calibration points; Inputting the pixel coordinates of the preset area into the coordinate mapping model to obtain the three-dimensional coordinates of the preset area; According to the three-dimensional coordinates of the preset area, coordinate information of a plurality of target points in the checkerboard calibration plate having the same position as the color calibration plate in the world coordinate system is determined.

4. The method according to claim 3, characterized in that Calculating the intrinsic parameter matrix and the extrinsic parameter matrix of the distortion-free imaging model includes: Inputting coordinate information of a plurality of target points in the chessboard calibration plate in the world coordinate system and coordinate information of the target points in the pixel coordinate system into the undistorted imaging model to obtain a homography matrix; Determining the constraint equation corresponding to the symmetric matrix of the intrinsic parameter matrix according to the coordinate information of the target point of the checkerboard calibration plate in the plurality of environmental images in the pixel coordinate system; Based on the constraint equation, the symmetric matrix is ​​calculated using a fitting algorithm; Determine an internal parameter matrix corresponding to the symmetric matrix; The product of the inverse matrix of the intrinsic parameter matrix and the homography matrix is ​​used as the extrinsic parameter matrix.

5. The method according to claim 2, characterized in that: Determining the distortion parameters corresponding to the distortion fitting model by using a nonlinear fitting algorithm includes: Inputting coordinate information of a plurality of target points in the checkerboard calibration plate in the pixel coordinate system into the distortion fitting model to obtain a plurality of constraint equations constructed by distortion parameters; A plurality of constraint equations constructed by distortion parameters are fitted using a nonlinear fitting algorithm, and the distortion parameters are obtained according to the fitting results.

6. The method according to claim 5, characterized in that The distortion parameters, the internal parameter matrix and the external parameter matrix are used as decision variables to establish the objective function of the distortion fitting model under the regularization constraint condition, including: Taking the distortion parameters, the internal parameter matrix and the external parameter matrix as decision variables, determining the difference between the environment image before and after the distortion under the current decision variables and the regularization terms corresponding to the distortion parameters, the internal parameter matrix and the external parameter matrix; The objective function is established by performing a weighted summation on the difference degree and the regularization term.

7. The method according to claim 1, characterized in that The objective function is optimized by using a nonlinear optimization algorithm to optimize the distortion parameters, the intrinsic parameter matrix and the extrinsic parameter matrix, including: Take the current decision variable as the current iteration point; Constructing a similar function to the target function in the neighborhood of the current iteration point; Calculate the minimum value corresponding to the similarity function in the neighborhood, and use the minimum point corresponding to the minimum value as the next iteration point; Calculating the similarity between the target function and the similarity function according to the current iteration point and the next iteration point; If the similarity is greater than a first threshold, taking the next iteration point as the current iteration point; If the similarity is not greater than the first threshold, determining an iteration step length according to the similarity, and iterating the current iteration point using the iteration step length, so as to use the iteration result as the current iteration point; Repeat the above steps of updating the current iteration point until the objective function converges.

8. A target positioning method, characterized in that: The method comprises: Receiving an image to be positioned; the image to be positioned is generated by a camera calibrated using the method according to any one of claims 1 to 7; The image to be positioned is input into a pre-trained target positioning model to obtain a positioning result.

9. A camera calibration device, characterized in that: The device comprises: An acquisition module, used to acquire multiple environment images generated by the camera to be calibrated; An imaging modeling module, used to establish a distortion-free imaging model according to the perspective imaging process of the chessboard calibration plate in the environmental image; A parameter calculation module, used to calculate the intrinsic parameter matrix and the extrinsic parameter matrix of the distortion-free imaging model; A distortion modeling module, used to establish a distortion fitting model of the environment image; A distortion fitting module, used to determine the distortion parameters corresponding to the distortion fitting model using a nonlinear fitting algorithm for the environmental image before and after distortion; An objective function establishment module is used to establish an objective function of the distortion fitting model under regularity constraints by taking the distortion parameters, the internal parameter matrix and the external parameter matrix as decision variables; An optimization module, used for optimizing the objective function by using a nonlinear optimization algorithm to optimize the distortion parameters, the intrinsic parameter matrix and the extrinsic parameter matrix; The calibration module is used to complete the camera calibration according to the optimized intrinsic parameter matrix, extrinsic parameter matrix and distortion parameters.

10. A target positioning device, characterized in that: The device comprises: A receiving module, used for receiving an image to be positioned; the image to be positioned is generated by a camera calibrated by the method according to any one of claims 1 to 7; The positioning module is used to input the image to be positioned into a pre-trained target positioning model to obtain a positioning result.

11. A computer device comprising a memory, a processor, and a computer program stored in the memory, characterized in that: When the computer program is executed by the processor, the computer program executes the instructions of the method according to any one of claims 1 to 8.

12. A computer storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor of a computer device, the computer program executes the instructions of the method according to any one of claims 1 to 8.

13. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the instructions of the method according to any one of claims 1 to 8 are executed.

Citation Information

Patent Citations

  • Three-dimensional calibration target-based multi-fisheye camera calibration device and method

    CN106548477A

  • Fisheye camera calibration algorithm evaluation method

    CN106960456A

  • Camera calibration method, electronic equipment and storage medium

    CN115457147A

  • Parameter calibration method, device and equipment of binocular fisheye camera and storage medium

    CN115564842A

  • Camera calibration method and system, and computer readable storage medium

    CN116071433A

Cited By

  • Target positioning method and system based on machine vision

    CN120088333A

  • Spatial positioning method, spatial positioning device and computer equipment

    CN121458801A