A calibration method, device, equipment and storage medium of a calibration system

Abstract

The application discloses a calibration method, device and equipment of a calibration system and a storage medium, and belongs to the field of computer vision. The method comprises the following steps: moving a first calibration plate in the shooting direction of a camera to be calibrated; if a calibration plate switching event is identified, generating a calibration plate switching instruction, and switching from the first calibration plate to a second calibration plate; and determining the calibration parameters of the camera to be calibrated according to the imaging results of at least two calibration plates. According to the technical scheme, calibration plates of different sizes are arranged, and different sizes of calibration plates are switched under certain conditions to calibrate the optical positioning system, so that the problem of poor positioning accuracy of the mark points of the calibration plate at a long distance in the prior art is solved, the positioning accuracy of the mark points in different field of view ranges is basically consistent, and the accuracy of the camera calibration parameters is improved.

Description

Technical Field

[0001] This application belongs to the field of computer vision, and specifically relates to a calibration method, apparatus, device and storage medium for a calibration system. Background Technology

[0002] In the field of computer vision, cameras are the primary carriers of information about the three-dimensional world, storing the obtained information in a matrix format in computers through video and image processing. However, in practice, camera sensors often cannot be used directly and require calibration before use. The acquisition of internal and external parameters directly determines the measurement and positioning accuracy. The process of determining these parameters using calibration devices and methods is called camera calibration. How to accurately calibrate cameras has always been a research hotspot in this field.

[0003] In existing technologies, camera calibration is typically based on binocular vision positioning systems. Specifically, this involves using a calibration plate of fixed size for camera calibration. However, when the field of view changes, the positioning accuracy of marker points on the fixed-size calibration plate is poor at longer distances. Therefore, providing a calibration system and method that ensures consistent positioning accuracy of marker points across different fields of view is a pressing issue in this technical field. Summary of the Invention

[0004] This application provides a calibration method, apparatus, device, and storage medium for a calibration system. By setting calibration plates of different sizes and providing set conditions to switch between using calibration plates of different sizes to calibrate the optical positioning system, it solves the problem of poor positioning accuracy of marker points at long distances in the prior art, and makes the positioning accuracy of marker points in different fields of view basically consistent, thereby improving the accuracy of camera calibration parameters.

[0005] In a first aspect, embodiments of this application provide a calibration method for a calibration system, the method comprising:

[0006] Control the first calibration plate to move in the shooting direction of the camera to be calibrated;

[0007] If a calibration board switching event is detected, a calibration board switching command is generated to switch from the first calibration board to the second calibration board;

[0008] The calibration parameters of the camera to be calibrated are determined based on the imaging results of the camera on at least two calibration plates.

[0009] Furthermore, if a calibration board switching event is detected, a calibration board switching command is generated to switch from the first calibration board to the second calibration board, including:

[0010] If the distance between the first calibration board and the camera to be calibrated is detected to reach a preset distance, a calibration board switching event is determined to have occurred; a calibration board switching command is generated to switch from the first calibration board to the second calibration board;

[0011] or,

[0012] If the marking point of the first calibration board is detected to achieve a preset imaging effect on the camera to be calibrated, a calibration board switching event is determined to have occurred; a calibration board switching command is generated to switch from the first calibration board to the second calibration board.

[0013] Furthermore, the first calibration plate moves from near to far in the shooting direction of the camera to be calibrated;

[0014] Correspondingly, if the distance between the first calibration board and the camera to be calibrated is detected to reach a preset distance, a calibration board switching event is determined to have occurred, including:

[0015] If the distance between the first calibration board and the camera to be calibrated is detected to exceed a preset distance, a calibration board switching event is determined to have occurred.

[0016] Correspondingly, if the marking points of the first calibration board are detected to achieve a preset imaging effect in the imaging effect of the camera to be calibrated, a calibration board switching event is determined to have occurred, including:

[0017] If the number of pixels occupied by the markers of the first calibration board in the image of the camera to be calibrated is less than the preset number of pixels, then a calibration board switching event is determined to have occurred.

[0018] Furthermore, the first calibration plate moves from far to near in the shooting direction of the camera to be calibrated;

[0019] Correspondingly, if the distance between the first calibration board and the camera to be calibrated is detected to reach a preset distance, a calibration board switching event is determined to have occurred, including:

[0020] If the distance between the first calibration board and the camera to be calibrated is detected to be less than a preset distance, a calibration board switching event is determined to have occurred.

[0021] Correspondingly, if the marking points of the first calibration board are detected to achieve a preset imaging effect in the imaging effect of the camera to be calibrated, a calibration board switching event is determined to have occurred, including:

[0022] If the number of pixels occupied by the markers of the first calibration board in the image of the camera to be calibrated exceeds the preset number of pixels, a calibration board switching event is determined to have occurred.

[0023] Furthermore, based on the imaging results of the camera to be calibrated on at least two calibration boards, the calibration parameters of the camera to be calibrated are determined, including:

[0024] Based on the imaging results of the camera to be calibrated on the first calibration board, the order of each marker point on the first calibration board is determined; and based on the imaging results of the camera to be calibrated on the second calibration board, the order of each marker point on the second calibration board is determined.

[0025] The pixel coordinates of each marker point are identified according to the order of the marker points;

[0026] The calibration parameters of the camera to be calibrated are determined based on the pixel coordinates and the spatial coordinates of each marker point.

[0027] Furthermore, based on the imaging results of the camera to be calibrated on the first calibration board, the order of each marker point on the first calibration board is determined; and based on the imaging results of the camera to be calibrated on the second calibration board, the order of each marker point on the second calibration board is determined, including:

[0028] Acquire the first calibration board image obtained by the camera to be calibrated, and identify the marker points in the first calibration board image;

[0029] The marker points in the first calibration board image are processed into a triangular mesh to obtain a triangular image with each marker point as a vertex;

[0030] Based on the triangle image, determine the order of the marker points in the first calibration plate image;

[0031] as well as,

[0032] Acquire the second calibration board image obtained by the camera to be calibrated, and identify the marker points in the second calibration board image;

[0033] The marker points in the second calibration board image are meshed into triangles to obtain a triangle image with each marker point as a vertex;

[0034] Based on the triangle image, determine the order of the marker points in the second calibration plate image.

[0035] Furthermore, based on the triangular image, determining the order of the marker points in the first calibration plate image includes:

[0036] Identify the angle value of the current marked point as a vertex;

[0037] If the sum of the angle values ​​is within a first range, then the current marker point is determined to be a corner point in the first calibration board image; if the sum of the angle values ​​is within a second range, then the current marker point is determined to be an edge point in the first calibration board image; if the sum of the angle values ​​is within a third range, then the current marker point is determined to be an interior point in the first calibration board image.

[0038] Determining the order of the marker points in the second calibration plate image based on the triangle image includes:

[0039] Identify the angle value of the current marked point as a vertex;

[0040] If the sum of the angle values ​​is within a first range, the current marker point is determined to be a corner point in the second calibration board image; if the sum of the angle values ​​is within a second range, the current marker point is determined to be an edge point in the second calibration board image; if the sum of the angle values ​​is within a third range, the current marker point is determined to be an interior point in the second calibration board image.

[0041] Secondly, embodiments of this application provide a calibration system for a calibration system, the system comprising:

[0042] The camera to be calibrated, at least two calibration boards, and control equipment.

[0043] Thirdly, embodiments of this application provide a calibration apparatus for a calibration system, the apparatus comprising:

[0044] The first calibration board control module is used to control the movement of the first calibration board in the shooting direction of the camera to be calibrated;

[0045] The switching event recognition module, if it recognizes a calibration board switching event, generates a calibration board switching command to switch from the first calibration board to the second calibration board;

[0046] The parameter calibration module is used to determine the calibration parameters of the camera to be calibrated based on the imaging results of the camera to be calibrated on at least two calibration plates.

[0047] Fourthly, embodiments of this application provide a control device, which includes a processor, a memory, and a program or instructions stored in the memory and executable on the processor. When the program or instructions are executed by the processor, they implement the steps of the calibration method of the calibration system as described in the first aspect.

[0048] Fifthly, embodiments of this application provide a readable storage medium storing a program or instructions that, when executed by a processor, implement the steps of the calibration method of the calibration system as described in the first aspect.

[0049] In this embodiment, the first calibration plate is controlled to move in the shooting direction of the camera to be calibrated; if a calibration plate switching event is detected, a calibration plate switching command is generated, switching from the first calibration plate to the second calibration plate; based on the imaging results of the camera to be calibrated on at least two calibration plates, the calibration parameters of the camera to be calibrated are determined. This technical solution, by setting calibration plates of different sizes and providing set conditions to switch between using calibration plates of different sizes to calibrate the optical positioning system, solves the problem of poor positioning accuracy of marker points at long distances in the prior art, making the positioning accuracy of marker points within different fields of view basically consistent, thereby improving the accuracy of camera calibration parameters. Attached Figure Description

[0050] Figure 1a This is a schematic flowchart of the calibration method of the calibration system provided in Embodiment 1 of this application;

[0051] Figure 1b This is a schematic diagram showing the arrangement of marker points on the calibration plate in an embodiment of this application;

[0052] Figure 2 This is a schematic flowchart of the calibration method of the calibration system provided in Embodiment 2 of this application;

[0053] Figure 3a This is a schematic flowchart of the calibration method of the calibration system provided in Embodiment 3 of this application;

[0054] Figure 3b This is a schematic diagram of triangular meshing in an embodiment of this application;

[0055] Figure 3c This is a schematic diagram of an embodiment of the present application, in which the marked points are vertices, edge points, and interior points;

[0056] Figure 3d This is a schematic diagram of an embodiment of the present application including a starting marker point and marker points A, B, C, and D as vertex corners;

[0057] Figure 3e This is a schematic diagram of a pinhole perspective projection model according to an embodiment of this application;

[0058] Figure 4 This is a schematic diagram of the calibration device of the calibration system provided in Embodiment 4 of this application;

[0059] Figure 5 This is a schematic diagram of the control device provided in Embodiment 5 of this application. Detailed Implementation

[0060] To make the objectives, technical solutions, and advantages of this application clearer, specific embodiments of this application will be described in further detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are merely for explaining this application and not for limiting it. It should also be noted that, for ease of description, only the parts relevant to this application are shown in the drawings, not all of them. Before discussing exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe operations (or steps) as sequential processes, many of these operations can be performed in parallel, concurrently, or simultaneously. Furthermore, the order of the operations can be rearranged. The process can be terminated when its operation is completed, but may also have additional steps not included in the drawings. The process can correspond to a method, function, procedure, subroutine, subprogram, etc.

[0061] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0062] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0063] The calibration method, apparatus, equipment, and storage medium of the calibration system provided in this application will be described in detail below with reference to the accompanying drawings and through specific embodiments and application scenarios.

[0064] Example 1

[0065] Figure 1a This is a schematic flowchart of the calibration method for the calibration system provided in Embodiment 1 of this application. The calibration system includes a camera to be calibrated, at least two calibration boards, and a control device; the calibration boards are provided with marker points; the method is executed by the control device. Figure 1a As shown, the specific steps include the following:

[0066] S101, Control the first calibration plate to move in the shooting direction of the camera to be calibrated.

[0067] First, one application scenario for this embodiment is using a calibration board to calibrate camera parameters. Based on the above application scenario, it can be understood that the executing entity of this application can be a smart terminal, such as a smartphone, tablet computer, or desktop computer.

[0068] In this embodiment, the camera to be calibrated can be understood as the target camera that needs to be calibrated. It is understood that the camera to be calibrated includes at least two cameras. During shooting, the cameras are set with the same shooting parameters. The calibration board can be understood as a geometric model with an array of marker points at fixed intervals. The marker points in the same calibration board can be circular points of equal size and spacing used to map world coordinates to pixel coordinates. Figure 1b This is a schematic diagram of the arrangement of marker points in the calibration plate according to an embodiment of this application. Both the first and second calibration plates are provided with 7×7+1 marker points. The specific arrangement of the marker points is as follows: Figure 1b As shown. The control device can be understood as an execution device used to perform the calibration method.

[0069] Understandably, the diameters of the markers on the two calibration plates are different. Specifically, the smaller marker diameter could be 7 mm. If the diameter is too small, the reflective brightness will be insufficient, making it difficult to identify the calibration plate when it is far from the camera. The larger marker diameter could be 15 mm. Considering the manufacturing process, a diameter exceeding 15 mm would result in higher production costs and requirements, and the calibration plate would be too heavy, making it less convenient to place during calibration.

[0070] The shooting direction of the camera to be calibrated can be the direction that the camera lens of the camera to be calibrated is facing. The first calibration plate moves in a uniform linear motion in the shooting direction of the camera to be calibrated.

[0071] In this embodiment, the robotic arm controls the first calibration plate to move at a constant speed in a straight line in the direction directly opposite the camera of the camera to be calibrated.

[0072] S102. If a calibration board switching event is detected, a calibration board switching command is generated to switch from the first calibration board to the second calibration board.

[0073] In this embodiment, the calibration board switching event can be either the control device detecting that the calibration board meets preset switching conditions, or the control device receiving a calibration board switching request sent by a technician. The generated calibration board switching instruction can be understood as control code containing calibration board switching information. The second calibration board can be understood as the one with the larger mark diameter among the two calibration boards. The switching from the first calibration board to the second calibration board can be achieved by controlling a robotic arm, or it can be done manually by a technician.

[0074] In this embodiment, if the control device detects that the calibration board meets the preset switching conditions, or receives a calibration board switching request sent by the technician, it generates a control command to control the switching of the calibration board, and the robotic arm completes the switching from the first calibration board to the second calibration board based on the control command.

[0075] S103. Determine the calibration parameters of the camera to be calibrated based on the imaging results of the camera to be calibrated on at least two calibration plates.

[0076] In this embodiment, the imaging results of the camera to be calibrated on at least two calibration plates can be understood as images captured by the camera of two or more calibration plates during uniform motion. The camera parameters can include internal and external parameters. Internal parameters can be parameters related to the camera's own characteristics, such as focal length and pixel size; external parameters can be parameters in the world coordinate system, such as the camera's position and rotation direction. Determining the calibration parameters of the camera to be calibrated can be based on the imaging results of the calibration plates, completing the transformation between different coordinate systems of the marked points, thereby determining the camera's calibration parameters.

[0077] In this embodiment, the control device determines the camera's calibration parameters by transforming the coordinate systems of the marker points based on images captured by the camera of two or more calibration plates during uniform motion.

[0078] The technical solution provided in this embodiment controls the movement of a first calibration plate in the shooting direction of the camera to be calibrated; if a calibration plate switching event is detected, a calibration plate switching command is generated, switching from the first calibration plate to the second calibration plate; based on the imaging results of the camera to be calibrated on at least two calibration plates, the calibration parameters of the camera to be calibrated are determined. This technical solution, by setting calibration plates of different sizes and providing set conditions to switch between using calibration plates of different sizes to calibrate the optical positioning system, solves the problem of poor positioning accuracy of marker points at long distances in the prior art, making the positioning accuracy of marker points within different fields of view basically consistent, thereby improving the accuracy of camera calibration parameters.

[0079] Example 2

[0080] Figure 2 This is a flowchart illustrating the calibration method of the calibration system provided in Embodiment 2 of this application. The execution of steps 202 and 203 is based on which preset condition is triggered first. For example... Figure 2 As shown, the specific steps include the following:

[0081] S201. Control the first calibration plate to move in the shooting direction of the camera to be calibrated.

[0082] S202. If it is detected that the distance between the first calibration board and the camera to be calibrated reaches a preset distance, a calibration board switching event is determined to have occurred; a calibration board switching command is generated to switch from the first calibration board to the second calibration board.

[0083] In this embodiment, the distance between the first calibration plate and the camera to be calibrated reaching a preset distance can be either greater than the preset distance, or less than or equal to the preset distance. The preset distance is set by technicians based on rigorous calculations.

[0084] In this embodiment, if the control device detects that the distance between the first calibration board and the camera to be calibrated is greater than a preset distance, it determines that the calibration board switching condition is met and generates a control command that controls the switching of the calibration board. The robotic arm then completes the switching from the first calibration board to the second calibration board based on the control command. It is understood that during the movement of the calibration board, the positioning accuracy of the marker points on the calibration board is poor. In this case, switching the calibration board achieves accurate positioning of the marker points.

[0085] S203. If the imaging effect of the first calibration board's marker point on the camera to be calibrated reaches the preset imaging effect, then a calibration board switching event is determined to have occurred; a calibration board switching command is generated to switch from the first calibration board to the second calibration board.

[0086] In this embodiment, the preset imaging effect of the first calibration plate's markers on the camera to be calibrated can be achieved by the number of markers on the first calibration plate clearly captured by the camera being less than a preset number, or by the clarity of the markers being less than a preset clarity.

[0087] In this embodiment, if the control device detects that the number of marker points on the first calibration board clearly captured by the camera is less than a preset number, or the clarity of the marker points is less than a preset clarity, it determines that the calibration board switching condition is met and generates a control command that controls the switching of the calibration board. The robotic arm completes the switching from the first calibration board to the second calibration board based on the control command.

[0088] Based on the above S202 and S203, optionally, the first calibration plate moves from near to far in the shooting direction of the camera to be calibrated;

[0089] Correspondingly, if the distance between the first calibration board and the camera to be calibrated is detected to reach a preset distance, a calibration board switching event is determined to have occurred, including:

[0090] If the distance between the first calibration board and the camera to be calibrated is detected to exceed a preset distance, a calibration board switching event is determined to have occurred.

[0091] Correspondingly, if the marking points of the first calibration board are detected to achieve a preset imaging effect in the imaging effect of the camera to be calibrated, a calibration board switching event is determined to have occurred, including:

[0092] If the number of pixels occupied by the markers of the first calibration board in the image of the camera to be calibrated is less than the preset number of pixels, then a calibration board switching event is determined to have occurred.

[0093] In this embodiment, when the first calibration plate moves from near to far in the shooting direction of the camera to be calibrated, the first calibration plate is a calibration plate with a smaller diameter of the marker points, and the second calibration plate is a calibration plate with a larger diameter. The fact that the number of pixels occupied by the marker points in the image of the camera to be calibrated is less than a preset number of pixels can be understood as indicating lower image quality and a blurry image.

[0094] In this embodiment, when the control device detects that the distance between the first calibration board and the camera to be calibrated is greater than a preset distance, or identifies that the number of pixels occupied by the marker points of the first calibration board in the image of the camera to be calibrated is less than a preset number of pixels, it determines that the switching conditions of the calibration board are met.

[0095] Understandably, at this time, if the distance between the first calibration board and the camera to be calibrated is greater than or equal to a preset distance, or if the number of pixels occupied by the markers on the first calibration board in the image of the camera to be calibrated is less than a preset number of pixels, the calibration board will be switched to the second calibration board with a larger marker diameter because the positioning accuracy of the markers on the first calibration board with a smaller marker diameter is poor.

[0096] The technical solution provided in this embodiment involves using a first calibration plate with a smaller marker diameter when the first calibration plate moves from near to far in the shooting direction of the camera to be calibrated. When the control device detects that the distance between the first calibration plate and the camera to be calibrated is greater than or equal to a preset distance, or identifies that the number of pixels occupied by the markers on the first calibration plate in the image of the camera to be calibrated is less than a preset number of pixels, it switches to a second calibration plate with a larger diameter. This ensures accurate positioning of the markers as the calibration plate moves from near to far in the shooting direction of the camera to be calibrated.

[0097] Based on the above S202 and S203, optionally, the first calibration plate moves from far to near in the shooting direction of the camera to be calibrated;

[0098] Correspondingly, if the distance between the first calibration board and the camera to be calibrated is detected to reach a preset distance, a calibration board switching event is determined to have occurred, including:

[0099] If the distance between the first calibration board and the camera to be calibrated is detected to be less than a preset distance, a calibration board switching event is determined to have occurred.

[0100] Correspondingly, if the marking points of the first calibration board are detected to achieve a preset imaging effect in the imaging effect of the camera to be calibrated, a calibration board switching event is determined to have occurred, including:

[0101] If the number of pixels occupied by the markers of the first calibration board in the image of the camera to be calibrated exceeds the preset number of pixels, a calibration board switching event is determined to have occurred.

[0102] In this embodiment, when the first calibration plate moves from far to near in the shooting direction of the camera to be calibrated, the first calibration plate is a calibration plate with a larger diameter of the marker points, and the second calibration plate is a calibration plate with a smaller diameter. If the number of pixels occupied by the marker points of the first calibration plate in the image of the camera to be calibrated exceeds a preset number of pixels, it may result in excessively high pixel density and low image quality.

[0103] In this embodiment, when the control device detects that the distance between the first calibration board and the camera to be calibrated is less than a preset distance, or identifies that the number of pixels occupied by the marker points of the first calibration board in the image of the camera to be calibrated is greater than or equal to a preset number of pixels, it determines that the switching conditions of the calibration board are met.

[0104] Understandably, at this time, if the distance between the first calibration board and the camera to be calibrated is less than a preset distance, or if the number of pixels occupied by the markers on the first calibration board in the image of the camera to be calibrated is greater than or equal to a preset number of pixels, the calibration board is switched to a second calibration board with a smaller marker diameter because the positioning accuracy of the markers on the first calibration board with a larger marker diameter is poor.

[0105] The technical solution provided in this embodiment involves using a first calibration plate with a larger marker diameter when the first calibration plate moves from far to near in the shooting direction of the camera to be calibrated. When the control device detects that the distance between the first calibration plate and the camera to be calibrated is less than a preset distance, or identifies that the number of pixels occupied by the markers on the first calibration plate in the image of the camera to be calibrated is greater than or equal to a preset number of pixels, it switches to a second calibration plate with a smaller diameter. This ensures accurate positioning of the markers as the calibration plate moves from far to near in the shooting direction of the camera to be calibrated.

[0106] S204. Determine the calibration parameters of the camera to be calibrated based on the imaging results of the camera to be calibrated on at least two calibration plates.

[0107] The technical solution provided in this embodiment sets different calibration plate switching conditions based on the different movement directions of the first calibration plate, which can achieve accurate positioning of the calibration plate's marker points in different movement directions.

[0108] Example 3

[0109] Figure 3a This is a flowchart illustrating the calibration method of the calibration system provided in Embodiment 2 of this application. Figure 3a As shown, the specific steps include the following:

[0110] S301, Control the first calibration plate to move in the shooting direction of the camera to be calibrated.

[0111] S302. If a calibration board switching event is detected, a calibration board switching command is generated to switch from the first calibration board to the second calibration board.

[0112] S303. Based on the imaging results of the camera to be calibrated on the first calibration board, determine the order of each marker point on the first calibration board; and based on the imaging results of the camera to be calibrated on the second calibration board, determine the order of each marker point on the second calibration board.

[0113] Determining the order of each marker point can be understood as determining the order of the marker points of the four vertices in the calibration board.

[0114] In this embodiment, the control device determines the order of the marker points of the four vertices in the first calibration board and the second calibration board based on the imaging results of the camera to be calibrated on the first calibration board and the imaging results on the second calibration board, respectively.

[0115] In this embodiment, optionally, determining the order of each marker point on the first calibration board based on the imaging result of the camera to be calibrated on the first calibration board, and determining the order of each marker point on the second calibration board based on the imaging result of the camera to be calibrated on the second calibration board, includes:

[0116] Acquire the first calibration board image obtained by the camera to be calibrated, and identify the marker points in the first calibration board image;

[0117] The marker points in the first calibration board image are processed into a triangular mesh to obtain a triangular image with each marker point as a vertex;

[0118] Based on the triangle image, determine the order of the marker points in the first calibration plate image;

[0119] as well as,

[0120] Acquire the second calibration board image obtained by the camera to be calibrated, and identify the marker points in the second calibration board image;

[0121] The marker points in the second calibration board image are meshed into triangles to obtain a triangle image with each marker point as a vertex;

[0122] Based on the triangle image, determine the order of the marker points in the second calibration plate image.

[0123] In this embodiment, Figure 3b This is a schematic diagram of triangular meshing in an embodiment of this application. For example... Figure 3b As shown, the triangular meshing can be understood as each marker point being connected to two adjacent marker points to form a triangle with each marker point as its vertex.

[0124] Specifically, the control device reads the first calibration board image captured by the camera to be calibrated and identifies the marker points in the first calibration board image. Specifically, this can be done by processing the first calibration board image into grayscale and determining the marker points based on the grayscale values. The marker points in the first calibration board image are then processed into a triangular mesh to obtain a triangular image with each marker point as a vertex. Based on the triangular image, the order of the four vertex marker points in the first calibration board image is determined. The same processing method is used to determine the order of the four vertex marker points in the second calibration board image.

[0125] In this embodiment, optionally, determining the order of the marker points in the first calibration plate image based on the triangle image includes:

[0126] Identify the angle value of the current marked point as a vertex;

[0127] If the sum of the angle values ​​is within a first range, then the current marker point is determined to be a corner point in the first calibration board image; if the sum of the angle values ​​is within a second range, then the current marker point is determined to be an edge point in the first calibration board image; if the sum of the angle values ​​is within a third range, then the current marker point is determined to be an interior point in the first calibration board image.

[0128] Determining the order of the marker points in the second calibration plate image based on the triangle image includes:

[0129] Identify the angle value of the current marked point as a vertex;

[0130] If the sum of the angle values ​​is within a first range, the current marker point is determined to be a corner point in the second calibration board image; if the sum of the angle values ​​is within a second range, the current marker point is determined to be an edge point in the second calibration board image; if the sum of the angle values ​​is within a third range, the current marker point is determined to be an interior point in the second calibration board image.

[0131] Figure 3c This is a schematic diagram illustrating embodiments of this application that include markers as vertices, edge points, and interior points. For example... Figure 3c As shown, the vertices can be understood as markers arranged at the four corners of the rectangle, like marker 1. The edge points can be understood as markers arranged along the edge lines of the rectangle, like marker 2. The interior points can be understood as markers arranged inside the rectangle, like marker 3.

[0132] In this embodiment, the angle values ​​of each marked point as a vertex of a triangle in the first calibration board image are determined sequentially. Specifically, the angle values ​​of each triangle corresponding to the marked point are added together, assuming the sum is M. For example, when 85°≤M≤95°, the current marked point is determined to be a corner point in the first calibration board image. When 175°≤M≤185°, the current marked point is determined to be an edge point in the first calibration board image. When 355°≤M≤365°, the current marked point is determined to be an interior point in the second calibration board image. Corner points, edge points, and interior points in the second calibration board image are determined in the same way.

[0133] Figure 3d This is a schematic diagram of an embodiment of this application including a starting marker point and marker points A, B, C, and D as vertex corners. For example... Figure 3d As shown, the distances between the starting marker point and the four corner marker points A, B, C, and D are calculated respectively. According to geometric principles, the point closest to the starting marker point is corner point A, which is designated as the first corner point. The point farthest from the starting marker point is corner point C, which is designated as the third corner point according to the clockwise marking order. Based on the first corner point, the vector angles between the first corner point and the other two corner points are calculated; the smaller angle is designated as the second corner point, and the larger angle as the fourth corner point. This operation is performed on both the first and second calibration board images to obtain the spatial coordinates corresponding to each marker point.

[0134] The technical solution provided in this embodiment performs triangular meshing on the marker points in the calibration board to obtain a triangular image with each marker point as a vertex. The distances between the starting marker point and the four vertex marker points, as well as the vector angles between the first corner point and the other two corner points, are calculated to determine the order of the four vertex marker points in the calibration board image, thereby quickly determining the spatial coordinates of each marker point and improving the efficiency of spatial coordinate determination.

[0135] S304. Identify the pixel coordinates of each marker point according to the order of the marker points.

[0136] In this embodiment, the pixel coordinates can be understood as the coordinate information of the marker point in the coordinate system composed of all pixels. The control device determines the specific position of each vertex in the image based on the order of the vertex marker points in the calibration board, and determines the pixel coordinates of each vertex marker point based on its position.

[0137] S305. Determine the calibration parameters of the camera to be calibrated based on the pixel coordinates and the spatial coordinates of each marker point.

[0138] In this embodiment, the spatial coordinates can be understood as the coordinate information of each marker point in a coordinate system established with one of the marker points in the calibration plate as the origin. Specifically, based on the pixel coordinates and the spatial coordinates of each marker point, the calibration parameters of the camera to be calibrated can be calculated using a perspective projection model based on the pinhole imaging principle. Figure 3e This is a schematic diagram of a pinhole perspective projection model according to an embodiment of this application. The pinhole perspective projection model is as follows: Figure 3e As shown.

[0139] Wherein, P(X) w ,Y w Z w ) is a three-dimensional coordinate system, O c X c Y c Z c It is the camera coordinate system, o i xy is the image coordinate system. Assume the coordinates of point P in the camera coordinate system are (X, Y) c ,Y c Z c ),go through Figure 3e The projection model shown has spatial coordinates p(x,y) projected onto the image. Based on the scaling relationship of pinhole imaging, we have:

[0140]

[0141] Transform equation (1) into a matrix representation of homogeneous coordinates:

[0142]

[0143] The relationship between the coordinates (u,v) of any pixel in the image and its spatial coordinates (x,y) is as follows:

[0144]

[0145] Transform equation (3) into a representation of homogeneous coordinates and matrices:

[0146]

[0147] There exists a rotation matrix R and a translation vector T between the world coordinate system and the camera coordinate system. Assume a point in space P(X... w ,Y w Z w If its coordinates in the camera coordinate system are given by..., then its coordinates in the camera coordinate system can be expressed as...

[0148]

[0149] Here, R is a 3×3 orthogonal matrix, and T is a 3×1 vector. Its homogeneous coordinates are represented as:

[0150]

[0151] Combining equations (1) to (6), the relationship between P and its pixel coordinates on the image can be obtained as follows:

[0152]

[0153] Let f x =f / d x f y =f / d y These are the focal lengths of the camera along the x and y axes of the image pixel coordinates, expressed in pixels. When d x =d y When, then f x =f y From the formula, we can see that f x f y u0 and v0 are determined by the camera's internal structure, hence they are called the camera's internal parameters. Therefore, the camera's internal parameter matrix... Here, s is the camera's tilt factor, i.e., the angle between the two vertical axes of the imaging plane. When the axial deflection of the coordinate axes on the imaging plane is not considered, s = 0. Substituting A into the equation, we get:

[0154]

[0155] R and T are the external parameters of the camera, which are related to the camera's orientation relative to the world coordinate system.

[0156] Given the diameter of the marker point and the pixel coordinates corresponding to the diameter of the marker point, the actual distance between the marker point and the camera can be calculated using Equation (8) to solve for the external and internal parameters of the camera to be calibrated, and the camera to be calibrated can be calibrated using the parameters.

[0157] The technical solution provided in this embodiment determines the spatial coordinates and pixel coordinates of the marker points on the first and second calibration boards under different visual ranges, and determines the camera calibration parameters based on the spatial coordinates and pixel coordinates of the marker points. This ensures that the positioning accuracy of the marker points is basically consistent under different visual ranges, further improving the accuracy of the camera parameters to be calibrated.

[0158] Example 4

[0159] Figure 4 This is a schematic diagram of the calibration device of the calibration system provided in Embodiment 3 of this application. As shown in Figure 3, it specifically includes the following:

[0160] The first calibration board control module 401 is used to control the first calibration board to move in the shooting direction of the camera to be calibrated;

[0161] If the switching event recognition module 402 recognizes a calibration board switching event, it generates a calibration board switching command to switch from the first calibration board to the second calibration board.

[0162] The parameter calibration module 403 is used to determine the calibration parameters of the camera to be calibrated based on the imaging results of the camera to be calibrated on at least two calibration plates.

[0163] The technical solution provided in this embodiment controls the movement of a first calibration plate in the shooting direction of the camera to be calibrated; if a calibration plate switching event is detected, a calibration plate switching command is generated, switching from the first calibration plate to the second calibration plate; based on the imaging results of the camera to be calibrated on at least two calibration plates, the calibration parameters of the camera to be calibrated are determined. This technical solution, by setting calibration plates of different sizes and providing set conditions to switch between using calibration plates of different sizes to calibrate the optical positioning system, solves the problem of poor positioning accuracy of marker points at long distances in the prior art, making the positioning accuracy of marker points within different fields of view basically consistent, thereby improving the accuracy of camera calibration parameters.

[0164] Furthermore, the switching event recognition module 402 includes a first recognition unit, which is used for:

[0165] If the distance between the first calibration board and the camera to be calibrated is detected to reach a preset distance, a calibration board switching event is determined to have occurred; a calibration board switching command is generated to switch from the first calibration board to the second calibration board;

[0166] or,

[0167] If the marking point of the first calibration board is detected to achieve a preset imaging effect on the camera to be calibrated, a calibration board switching event is determined to have occurred; a calibration board switching command is generated to switch from the first calibration board to the second calibration board.

[0168] The technical solution provided in this embodiment sets different calibration plate switching conditions based on the different movement directions of the first calibration plate, which can achieve accurate positioning of the calibration plate's marker points in different movement directions.

[0169] Furthermore, the switching event recognition module 402 also includes a second recognition unit. When the first calibration plate moves from near to far in the shooting direction of the camera to be calibrated, the second recognition unit is used to:

[0170] If the distance between the first calibration board and the camera to be calibrated is detected to exceed a preset distance, a calibration board switching event is determined to have occurred.

[0171] or,

[0172] If the number of pixels occupied by the markers of the first calibration board in the image of the camera to be calibrated is less than the preset number of pixels, then a calibration board switching event is determined to have occurred.

[0173] The technical solution provided in this embodiment involves using a first calibration plate with a smaller marker diameter when the first calibration plate moves from near to far in the shooting direction of the camera to be calibrated. When the control device detects that the distance between the first calibration plate and the camera to be calibrated is greater than or equal to a preset distance, or identifies that the number of pixels occupied by the markers on the first calibration plate in the image of the camera to be calibrated is less than a preset number of pixels, it switches to a second calibration plate with a larger diameter. This ensures accurate positioning of the markers as the calibration plate moves from near to far in the shooting direction of the camera to be calibrated.

[0174] Furthermore, the switching event recognition module 402 also includes a third recognition unit. When the first calibration plate moves from far to near in the shooting direction of the camera to be calibrated, the third recognition unit is used to:

[0175] If the distance between the first calibration board and the camera to be calibrated is detected to be less than a preset distance, a calibration board switching event is determined to have occurred.

[0176] or,

[0177] If the number of pixels occupied by the markers of the first calibration board in the image of the camera to be calibrated exceeds the preset number of pixels, a calibration board switching event is determined to have occurred.

[0178] The technical solution provided in this embodiment involves using a first calibration plate with a larger marker diameter when the first calibration plate moves from far to near in the shooting direction of the camera to be calibrated. When the control device detects that the distance between the first calibration plate and the camera to be calibrated is less than a preset distance, or identifies that the number of pixels occupied by the markers on the first calibration plate in the image of the camera to be calibrated is greater than or equal to a preset number of pixels, it switches to a second calibration plate with a smaller diameter. This ensures accurate positioning of the markers as the calibration plate moves from far to near in the shooting direction of the camera to be calibrated.

[0179] Furthermore, the parameter calibration module 403 is specifically used for:

[0180] Based on the imaging results of the camera to be calibrated on the first calibration board, the order of each marker point on the first calibration board is determined; and based on the imaging results of the camera to be calibrated on the second calibration board, the order of each marker point on the second calibration board is determined.

[0181] The pixel coordinates of each marker point are identified according to the order of the marker points;

[0182] The calibration parameters of the camera to be calibrated are determined based on the pixel coordinates and the spatial coordinates of each marker point.

[0183] The technical solution provided in this embodiment determines the spatial coordinates and pixel coordinates of the marker points on the first and second calibration boards under different visual ranges, and determines the camera calibration parameters based on the spatial coordinates and pixel coordinates of the marker points. This ensures that the positioning accuracy of the marker points is basically consistent under different visual ranges, further improving the accuracy of the camera parameters to be calibrated.

[0184] Furthermore, the calibration device of the calibration system also includes a marker point order determination module, which is used for:

[0185] Acquire the first calibration board image obtained by the camera to be calibrated, and identify the marker points in the first calibration board image;

[0186] The marker points in the first calibration board image are processed into a triangular mesh to obtain a triangular image with each marker point as a vertex;

[0187] Based on the triangle image, determine the order of the marker points in the first calibration plate image;

[0188] as well as,

[0189] Acquire the second calibration board image obtained by the camera to be calibrated, and identify the marker points in the second calibration board image;

[0190] The marker points in the second calibration board image are meshed into triangles to obtain a triangle image with each marker point as a vertex;

[0191] Based on the triangle image, determine the order of the marker points in the second calibration plate image.

[0192] Furthermore, the marker order determination module includes a first determination unit, which is used to:

[0193] Identify the angle value of the current marked point as a vertex;

[0194] If the sum of the angle values ​​is within a first range, then the current marker point is determined to be a corner point in the first calibration board image; if the sum of the angle values ​​is within a second range, then the current marker point is determined to be an edge point in the first calibration board image; if the sum of the angle values ​​is within a third range, then the current marker point is determined to be an interior point in the first calibration board image.

[0195] Determining the order of the marker points in the second calibration plate image based on the triangle image includes:

[0196] Identify the angle value of the current marked point as a vertex;

[0197] If the sum of the angle values ​​is within a first range, the current marker point is determined to be a corner point in the second calibration board image; if the sum of the angle values ​​is within a second range, the current marker point is determined to be an edge point in the second calibration board image; if the sum of the angle values ​​is within a third range, the current marker point is determined to be an interior point in the second calibration board image.

[0198] The technical solution provided in this embodiment performs triangular meshing on the marker points in the calibration board to obtain a triangular image with each marker point as a vertex. The distances between the starting marker point and the four vertex marker points, as well as the vector angles between the first corner point and the other two corner points, are calculated to determine the order of the four vertex marker points in the calibration board image, thereby quickly determining the spatial coordinates of each marker point and improving the efficiency of spatial coordinate determination.

[0199] The calibration device in the calibration system of this application embodiment can be a device, or a component, integrated circuit, or chip in a terminal. The device can be a mobile control device or a non-mobile control device. For example, a mobile control device can be a mobile phone, tablet computer, laptop computer, PDA, vehicle control device, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc. A non-mobile control device can be a server, network attached storage (NAS), personal computer (PC), television set (TV), ATM, or self-service machine, etc. This application embodiment does not impose specific limitations.

[0200] The calibration device of the calibration system in this application embodiment can be a device with an operating system. The operating system can be Android, iOS, or other possible operating systems, and this application embodiment does not specifically limit it.

[0201] The calibration device of the calibration system provided in this application embodiment can realize the various processes implemented in the above method embodiments. To avoid repetition, it will not be described again here.

[0202] Example 5

[0203] Figure 5 This is a schematic diagram of the control device provided in Embodiment 5 of this application. Figure 5 As shown, this application embodiment also provides a control device 500, including a processor 501, a memory 502, and a program or instructions stored in the memory 502 and executable on the processor 501. When the program or instructions are executed by the processor 501, they implement the various processes of the calibration method embodiment of the above-described calibration system and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0204] It should be noted that the control device in the embodiments of this application includes the mobile control device and the non-mobile control device described above.

[0205] Example 6

[0206] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the calibration method embodiment of the above-described calibration system and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0207] The processor is the processor in the control device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0208] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0209] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0210] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

[0211] The above description is merely a preferred embodiment and the technical principles employed in this application. This application is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions that can be made by those skilled in the art will not depart from the scope of protection of this application. Therefore, although this application has been described in detail through the above embodiments, this application is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of this application, the scope of which is determined by the scope of the claims.

Claims

1. A calibration method of a calibration system, characterized by, The calibration system comprises a camera to be calibrated, at least two calibration boards, and a control device; the calibration boards are provided with mark points; the method is executed by the control device, and the method comprises: controlling the first calibration board to move in the shooting direction of the camera to be calibrated; if a calibration board switching event is identified, generating a calibration board switching instruction to switch from the first calibration board to the second calibration board; obtaining a first calibration board image obtained by the camera to be calibrated, and identifying the mark points in the first calibration board image; triangular meshing the mark points in the first calibration board image to obtain triangular images with the mark points as vertices; determining the order of the mark points in the first calibration board image according to the triangular images; obtaining a second calibration board image obtained by the camera to be calibrated, and identifying the mark points in the second calibration board image; triangular meshing the mark points in the second calibration board image to obtain triangular images with the mark points as vertices; determining the order of the mark points in the second calibration board image according to the triangular images; identifying the pixel coordinates of the mark points according to the order of the mark points; determining the calibration parameters of the camera to be calibrated according to the pixel coordinates and the spatial coordinates of the mark points. if a calibration board switching event is identified, generating a calibration board switching instruction to switch from the first calibration board to the second calibration board, comprising:

2. The method of claim 1, wherein, if the distance between the first calibration board and the camera to be calibrated reaches a preset distance, it is determined that a calibration board switching event occurs; a calibration board switching instruction is generated to switch from the first calibration board to the second calibration board; or, if the imaging effect of the mark points of the first calibration board in the camera to be calibrated reaches a preset imaging effect, it is determined that a calibration board switching event occurs; a calibration board switching instruction is generated to switch from the first calibration board to the second calibration board. The first calibration board moves from near to far in the shooting direction of the camera to be calibrated; 3. The method of claim 2, wherein, Correspondingly, if the distance between the first calibration board and the camera to be calibrated reaches a preset distance, it is determined that a calibration board switching event occurs, comprising: if the distance between the first calibration board and the camera to be calibrated exceeds a preset distance, it is determined that a calibration board switching event occurs; Correspondingly, if the imaging effect of the mark points of the first calibration board in the camera to be calibrated reaches a preset imaging effect, it is determined that a calibration board switching event occurs, comprising: if the number of pixel points occupied by the mark points of the first calibration board in the imaging of the camera to be calibrated is less than a preset number of pixel points, it is determined that a calibration board switching event occurs. The first calibration board moves from far to near in the shooting direction of the camera to be calibrated; 4. The method of claim 2, wherein, Correspondingly, if the distance between the first calibration board and the camera to be calibrated reaches a preset distance, it is determined that a calibration board switching event occurs, comprising: if the distance between the first calibration board and the camera to be calibrated is less than a preset distance, it is determined that a calibration board switching event occurs; Correspondingly, if the imaging effect of the mark points of the first calibration board in the camera to be calibrated reaches a preset imaging effect, it is determined that a calibration board switching event occurs, comprising: ​ If the number of pixels occupied by the mark point of the first calibration plate in the imaging of the camera to be calibrated exceeds a preset number of pixels, it is determined that a calibration plate switching event occurs.

5. The method of claim 1, wherein, According to the triangular image, the order of the mark points in the first calibration plate image is determined, comprising: identifying the angle value of the current mark point as a vertex; if the sum of the angle values is within a first range, it is determined that the current mark point is a corner point in the first calibration plate image; if the sum of the angle values is within a second range, it is determined that the current mark point is an edge point in the first calibration plate image; if the sum of the angle values is within a third range, it is determined that the current mark point is an internal point in the first calibration plate image; According to the triangular image, the order of the mark points in the second calibration plate image is determined, comprising: identifying the angle value of the current mark point as a vertex; if the sum of the angle values is within a first range, it is determined that the current mark point is a corner point in the second calibration plate image; if the sum of the angle values is within a second range, it is determined that the current mark point is an edge point in the second calibration plate image; if the sum of the angle values is within a third range, it is determined that the current mark point is an internal point in the second calibration plate image.

6. A calibration device for calibrating a system, characterized in that The calibration system comprises a camera to be calibrated, at least two calibration plates, and a control device; the calibration plates are provided with mark points; the device is configured in the control device, and the device comprises: a first calibration plate control module for controlling the movement of the first calibration plate in the shooting direction of the camera to be calibrated; a switching event identification module, which generates a calibration plate switching instruction if a calibration plate switching event is identified, and switches from the first calibration plate to the second calibration plate; a parameter calibration module for obtaining the first calibration plate image obtained by the camera to be calibrated, and identifying the mark points in the first calibration plate image; triangular meshing is performed on the mark points in the first calibration plate image to obtain triangular images with each mark point as a vertex; according to the triangular image, the order of the mark points in the first calibration plate image is determined; and obtaining the second calibration plate image obtained by the camera to be calibrated, and identifying the mark points in the second calibration plate image; triangular meshing is performed on the mark points in the second calibration plate image to obtain triangular images with each mark point as a vertex; according to the triangular image, the order of the mark points in the second calibration plate image is determined; the pixel coordinates of the mark points are identified according to the order of the mark points; and the calibration parameters of the camera to be calibrated are determined according to the pixel coordinates and the spatial coordinates of the mark points.

7. A control device characterized by comprising: The readable storage medium stores programs or instructions, which are executed by the processor to implement the steps of the calibration method of the calibration system according to any one of claims 1-5.

8. A readable storage medium, characterized by, The readable storage medium stores programs or instructions, which are executed by the processor to implement the steps of the calibration method of the calibration system according to any one of claims 1-5.

Images