Camera calibration method and calibration board

Abstract

The embodiment of the invention discloses a camera calibration method and a calibration board, and belongs to the field of camera calibration. The camera calibration method comprises the following steps: acquiring a first image and a second image, and determining a first camera coordinate according to the first image; determining a second camera coordinate according to the second image; according to the first camera coordinate and the first calibration plate coordinate, obtaining a first pose relationship between the first camera and the calibration plate; according to the second camera coordinate and the second calibration plate coordinate, obtaining a second pose relationship between the second camera and the calibration plate; and obtaining a calibration result according to the first pose relationship and the second pose relationship. According to the embodiment of the invention, the method achieves the calibration of a plurality of cameras of which the shooting ranges are not overlapped.

Description

Technical Field

[0001] The present application relates to camera calibration, and more specifically, to a camera calibration method and a calibration board. Background Art

[0002] In the industrial production process, multiple cameras are used to capture products to obtain product images, and the product images are used to perform inspection, measurement and other processing on the products, which can improve the efficiency of product inspection, measurement, etc.

[0003] Before using multiple cameras to shoot a product, it is necessary to calibrate the multiple cameras so that the calibrated multiple cameras can share the product images taken by each camera. Take Camera 1 and Camera 2 as an example. If Camera 1 and Camera 2 are not calibrated, Camera 1 cannot know which position of the product the image taken by Camera 2 is because Camera 1 and Camera 2 are in different positions and / or shooting directions. Similarly, Camera 2 cannot know which position of the product the image taken by Camera 1 is. Therefore, only after calibration can the relative position relationship between Camera 1 and Camera 2 be determined, so that after obtaining the image taken by Camera 1, the image can be used in the calculation process of Camera 2 after position conversion, so as to realize joint detection or measurement between multiple cameras.

[0004] Most of the current calibration methods require limiting the overlapping fields of view between multiple cameras, and completing the calibration between multiple cameras through the overlapping parts. Therefore, there is an urgent need for a calibration method that can calibrate multiple cameras without overlapping fields of view. Summary of the invention

[0005] The embodiments of the present application provide a camera calibration method and a calibration board to at least solve the technical problem of being able to calibrate multiple cameras without overlapping fields of view.

[0006] According to a first aspect of an embodiment of the present application, a camera calibration method is provided, which is applied to at least two cameras, wherein the at least two cameras include a first camera and a second camera, wherein the shooting ranges of the first camera and the second camera have no overlapping area, the first camera has a first coordinate system, and the second camera has a second coordinate system, and the method includes:

[0007] Acquire a first image taken by the first camera for a preset first target and a second image taken by the second camera for a preset second target, wherein the first target and the second target are located on different sides of the same calibration plate, the calibration plate has a third coordinate system, the first target corresponds to first calibration plate coordinates in the third coordinate system, and the second target corresponds to second calibration plate coordinates in the third coordinate system, wherein the third coordinate system, the first calibration plate coordinates, and the second calibration plate coordinates are determined;

[0008] determining a first camera coordinate of the first target in the first coordinate system according to the first image;

[0009] determining a second camera coordinate of the second target in the second coordinate system according to the second image;

[0010] Obtaining a first pose relationship between the first camera and the calibration plate according to the first camera coordinates and the first calibration plate coordinates;

[0011] Obtaining a second position and posture relationship between the second camera and the calibration plate according to the second camera coordinates and the second calibration plate coordinates;

[0012] A calibration result of the first camera and the second camera is obtained according to the first posture relationship and the second posture relationship.

[0013] By adopting this embodiment, a calibration plate including different side faces can be utilized, so that the first camera can shoot a first target on one side face and finally obtain the first pose relationship between the first camera and the calibration plate, and the second camera can shoot a second target on the other side face and finally obtain the second pose relationship between the second camera and the calibration plate. Since the first pose relationship and the second pose relationship have the same reference object, i.e., the calibration plate, the calibration result can finally be obtained based on the first pose relationship and the second pose relationship, thereby realizing the calibration of multiple cameras with non-overlapping shooting ranges.

[0014] In combination with the first aspect, in an optional implementation manner of the embodiment of the present application, determining the first camera coordinates of the first target in the first coordinate system according to the first image includes:

[0015] Determining a first pixel coordinate of the first target according to first target information in the first image;

[0016] The first camera coordinates are determined according to the first pixel coordinates and camera parameters of the first camera.

[0017] By adopting this implementation method, the first pixel coordinates of the first target in the first image are determined, and then combined with the camera parameters to obtain the first camera coordinates. Since the first pixel coordinates and the camera parameters are both related to the first camera, more accurate first camera coordinates can be obtained efficiently.

[0018] In combination with the first aspect, in an optional implementation manner of the embodiment of the present application, the first target information includes a first edge point of the first target in the first image;

[0019] The determining the first pixel coordinates of the first target according to the first target information in the first image includes:

[0020] Calculating the gradient direction and intensity of the first edge point corresponding to each of the first targets to obtain a first initial center position of the corresponding first target;

[0021] Processing the first initial center position of each of the first targets using a least squares algorithm to obtain a first center position of each of the first targets in the first image;

[0022] The first center position is used as the first pixel coordinate.

[0023] With this implementation, the first initial center position is obtained using the gradient direction and intensity of the first edge point, and then the first center position is calculated using the least squares algorithm, which is equivalent to performing a preprocessing first and then a fine processing. This method helps to improve the accuracy of the first pixel coordinates.

[0024] In combination with the first aspect, in an optional implementation manner of the embodiment of the present application, determining the first camera coordinates according to the first pixel coordinates and camera parameters of the first camera includes:

[0025] The first camera coordinates are calculated based on the following formula:

[0026] V i =(f x ×(p ix -c x ),f y ×(p iy -c y ),1);

[0027] Among them, V i is the normal vector of the i-th first target in the first coordinate system, f x is the horizontal focal length of the first camera, f y is the vertical focal length of the first camera, c x is the horizontal coordinate of the principal point of the first camera, c y is the vertical coordinate of the principal point of the first camera, p ix is the horizontal coordinate of the i-th first target in the first coordinate system, p iy is the vertical coordinate of the i-th first target in the first coordinate system, and i is a positive integer.

[0028] By adopting this implementation, the first camera coordinates are calculated using a formula, which helps to improve the accuracy of the first camera coordinates and improve the calculation efficiency.

[0029] In combination with the first aspect, in an optional implementation method of an embodiment of the present application, the method of determining the second camera coordinates of the second target in the second coordinate system based on the second image is the same as the method of determining the first camera coordinates of the first target in the first coordinate system based on the first image.

[0030] In combination with the first aspect, in an optional implementation of the embodiment of the present application, the calibration plate is in a shape in which the length direction, the width direction and the height direction are perpendicular to each other;

[0031] The method further comprises:

[0032] The third coordinate system is obtained by taking the target position on the calibration plate as the coordinate origin and taking the length direction, width direction and height direction of the calibration plate as the directions of each coordinate axis.

[0033] With this implementation, once the structure of the calibration plate is determined, the target position can be determined. At the same time, since the length, width and height directions can all be determined, the third coordinate system can be obtained. That is, when the calibration plate is determined, the third coordinate system can also be determined.

[0034] In combination with the first aspect, in an optional implementation manner of the embodiment of the present application, the method further includes:

[0035] Determine the coordinates of the first calibration plate according to the distance between the first target and the target position in the length direction, the distance between the first target and the target position in the width direction, and the distance between the first target and the target position in the height direction;

[0036] The coordinates of the second calibration plate are determined according to the distance of the second target from the target position in the length direction, the distance of the second target from the target position in the width direction, and the distance of the second target from the target position in the height direction.

[0037] With this implementation, after the third coordinate system is determined, since the positions of the first target and the second target on the calibration plate are determined, the coordinates of the first calibration plate and the coordinates of the second calibration plate can be determined.

[0038] In combination with the first aspect, in an optional implementation of the embodiment of the present application, the first target is located on a first side of the calibration plate, and the second target is located on a second side of the calibration plate;

[0039] The method further comprises:

[0040] Determining the coordinates of a first calibration plate according to a positional relationship between the first side surface and the target position and a position of the first target on the first side surface;

[0041] The coordinates of the second calibration plate are determined according to the positional relationship between the second side surface and the target position and the position of the second target on the second side surface.

[0042] With this implementation, after the calibration plate is determined, the positional relationship between the first side surface, the second side surface and the target position can also be determined, and since the first target is on the first side surface, the first calibration plate coordinates can be obtained by the position of the first target on the first side surface. Similarly, the second calibration plate coordinates can be obtained.

[0043] In combination with the first aspect, in an optional implementation of the embodiment of the present application, the first side surface and the second side surface are the same side surface of the calibration plate, and the first target and the second target are located in different areas of the same side surface.

[0044] Alternatively, the first side surface and the second side surface are different sides of the calibration plate, and a distance between the first side surface and the second side surface enables the first camera and the second camera to be simultaneously located between the first side surface and the second side surface.

[0045] With this implementation, when the first side surface and the second side surface are different sides, it is helpful to calibrate the first camera and the second camera with different shooting directions. When the first side surface and the second side surface are the same side surface, since the first target and the second target are in different areas, it is helpful to calibrate the first camera and the second camera with a spacing distance but the same shooting direction.

[0046] In combination with the first aspect, in an optional implementation of the embodiment of the present application, the acquiring of a first image taken by the first camera for a preset first target and a second image taken by the second camera for a preset second target includes:

[0047] The first image and the second image are acquired at multiple positions of the calibration plate.

[0048] By adopting this implementation method, the amount of data is increased by acquiring the first image and the second image when the calibration plate has multiple postures, which helps to improve the calculation accuracy of the first posture relationship and the second posture relationship, thereby improving the accuracy of the calibration result.

[0049] In combination with the first aspect, in an optional implementation manner of the embodiment of the present application, the shooting directions of the first camera and the second camera are opposite.

[0050] By adopting this implementation, the first camera and the second camera with opposite shooting directions can be calibrated, thereby improving the application scope of the camera calibration method provided in this application.

[0051] In combination with the first aspect, in an optional implementation of the embodiment of the present application, obtaining the calibration results of the first camera and the second camera according to the first posture relationship and the second posture relationship includes:

[0052] The first posture relationship and the second posture relationship are calculated using the following formula to obtain the calibration result;

[0053]

[0054] in, To calibrate the results, is the transformation matrix from the third coordinate system to the first coordinate system during the i-th shooting, is the transformation matrix from the third coordinate system to the second coordinate system during the i-th shooting.

[0055] By adopting this implementation method, the calibration result is obtained by using the formula to calculate, which helps to improve the accuracy and calculation efficiency of the calibration result.

[0056] In combination with the first aspect, in an optional implementation manner of the embodiment of the present application, after obtaining the calibration results of the first camera and the second camera according to the first posture relationship and the second posture relationship, the method further includes:

[0057] The calibration result is processed by a second algorithm to obtain an optimization result, wherein the second algorithm includes an LM posture optimization algorithm, and the formula of the second algorithm includes:

[0058]

[0059] Where M is the number of shots, and its value range is a positive integer. R is the rotation matrix from the second coordinate system to the first coordinate system. T is the translation matrix from the second coordinate system to the first coordinate system. R i is the rotation matrix from the second coordinate system to the first coordinate system calculated during the i-th shooting, T i is the translation matrix from the second coordinate system to the first coordinate system calculated during the i-th shooting, E is the overall calculation error, and i is a positive integer.

[0060] By adopting this implementation method, the calibration results are optimized, which helps to improve the accuracy of the calibration results.

[0061] According to a second aspect of an embodiment of the present application, a calibration plate is provided for calibrating at least two cameras, the at least two cameras including a first camera and a second camera, the calibration plate including a first side wall, a second side wall and a connecting portion connecting the first side wall and the second side wall, the first side wall having a first side surface for setting a target, and the second side wall having a second side surface for setting a target;

[0062] The first side wall and the second side wall are respectively located at two ends of the connecting portion in the length direction;

[0063] At least one first target is disposed on the first side surface for being photographed by the first camera;

[0064] At least one second target is disposed on the second side surface for being photographed by the second camera;

[0065] In a third coordinate system based on the calibration plate, the first calibration plate coordinates of the first target and the second calibration plate coordinates of the second target are determined.

[0066] By adopting this implementation method, a calibration plate with different side surfaces is used, so that the first camera can shoot the first target on the first side to obtain the posture relationship between the first camera and the calibration plate, and the second camera can shoot the second target on the second side to obtain the posture relationship between the second camera and the calibration plate. Since there is the same reference object, i.e., the calibration plate, the calibration result can be finally obtained based on the two posture relationships, thereby realizing the calibration of multiple cameras with non-overlapping shooting ranges.

[0067] In combination with the second aspect, in an optional implementation of the embodiment of the present application, both the first side wall and the second side wall are rotatably connected to the connecting portion.

[0068] By adopting this implementation method, it is easy to adjust the posture between the first side wall and the first camera, and the posture between the second side wall and the second camera, thereby facilitating the acquisition of multiple sets of data and improving the calibration accuracy of the first camera and the second camera.

[0069] In combination with the second aspect, in an optional implementation of the embodiment of the present application, the calibration plate corresponds to the third coordinate system with the target position on the connecting portion as the coordinate origin and the length direction, width direction and height direction of the calibration plate as the directions of each coordinate axis;

[0070] The first calibration plate coordinates of the first target in the third coordinate system can be determined according to the positional relationship between the first side surface and the target position and the position of the first target on the first side surface;

[0071] The second calibration plate coordinates of the second target in the third coordinate system can be determined according to the positional relationship between the second side surface and the target position and the position of the second target on the second side surface.

[0072] In combination with the second aspect, in an optional implementation of the embodiment of the present application, the first side wall and the second side wall have the same size and shape, the connecting portion fixes the distance between the first side wall and the second side wall, the length direction of the connecting portion is perpendicular to the first side surface and the second side surface, and along the length direction of the connecting portion, the projection of the first side wall on the second side wall coincides with the second side wall.

[0073] The technical effect obtained by the third aspect mentioned above is similar to the technical effect obtained by the corresponding technical means in the first and second aspects, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] Figure 1 is a flowchart of a camera calibration method provided in an embodiment of the present application;

[0075] Figure 2 is a flow chart of determining the first camera coordinates in a camera calibration method provided in an embodiment of the present application;

[0076] Figure 3 is a schematic diagram of a camera calibration structure provided in an embodiment of the present application;

[0077] Figure 4 It is a flowchart for implementing the multi-viewpoint calibration method under opposite fields of view of the camera provided in an embodiment of the present application.

[0078] Figure 5 It is a schematic diagram of the principle of a monocular vision positioning method based on a returning light circular target provided in an embodiment of the present application.

[0079] Figure 6 It is a flow chart of the sub-pixel circle positioning algorithm provided in the embodiment of the present application.

[0080] Figure 7 It is a flow chart of the LM posture optimization algorithm provided in an embodiment of the present application.

[0081] Explanation of reference numerals: 1. Calibration plate; 11. First side wall; 12. Second side wall; 13. Connecting portion; 2. First target; 3. Second target; 4. First camera; 5. Second camera. DETAILED DESCRIPTION

[0082] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0083] It should be understood that the "plurality" mentioned in this article refers to two or more. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in order to facilitate the clear description of the technical solutions of the embodiments of the present application, in the embodiments of the present application, the words "first", "second" and so on are used to distinguish between the same items or similar items with basically the same functions and effects. Those skilled in the art can understand that the words "first", "second" and so on do not limit the quantity and execution order, and the words "first", "second" and so on do not limit certain different

[0084] In addition, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus that includes a series of steps or elements is not necessarily limited to those steps or elements explicitly listed, but may include other steps or elements not explicitly listed or inherent to such process, method, product, or apparatus.

[0085] First, the terms involved in the embodiments of the present application are introduced.

[0086] Camera calibration: is a method used to build a three-dimensional model, the main purpose of which is to determine the internal and external parameters of the camera. The method of the present invention refers to the calibration of external parameters, that is, the relationship between the camera coordinate system and the external coordinate system, where the external coordinate system refers to another camera coordinate system.

[0087] The purpose of camera calibration is to determine the relationship between the camera coordinate systems of two cameras. For the same point or object, if the position of the point in the coordinate system of camera A is known, the position of the point in the coordinate system of camera B can be known; if the pose (position plus pose) of the object in the coordinate system of camera A is known, the pose of the point in the coordinate system of camera B can be known; and the point or object does not need to be captured by camera B.

[0088] The consequence of not calibrating is that the two cameras are independent entities, and the information captured by camera A cannot be used by camera B. The two cameras cannot share coordinate information.

[0089] The object of calibration: the camera coordinate system of the two cameras. The result of calibration is the coordinate relationship between the two camera coordinate systems.

[0090] To give a simple example, an object is very long, with two points C and D on each end. At this time, the distance CD needs to be measured by visual methods. An ordinary camera A cannot capture both ends of CD at the same time, that is, it cannot measure the length of the object. If an additional camera is introduced, camera B, camera A captures point C, and camera B captures point D at the same time. If the relationship between the two coordinate systems AB has been calibrated, the coordinates of point C can be converted to the coordinate system of camera B, so the distance CD can be known, so that multiple cameras can be used to inspect products and other processing.

[0091] With the rapid development of industrial vision hardware, computer vision algorithms and industrial robots, vision and its controller act as the eyes and brains of robots, making robots play a more important role in industrial automation production lines. Through traditional image processing algorithms and deep learning algorithms, the vision controller can analyze and interpret the acquired image information in real time. This includes functions such as object recognition, position positioning, dimension measurement, and environmental map construction, and finally provides real-time feedback and intelligent decision-making capabilities for the robot. In the industrial production process, using multiple cameras to inspect and measure products can improve detection efficiency and accuracy and ensure product quality. Cameras with non-overlapping fields of view have a wide range of application scenarios in the industrial field, especially in 3D reconstruction, depth perception, and industrial product quality inspection.

[0092] In industrial product quality inspection, multiple cameras are installed according to the product features and inspection requirements to ensure that they can capture all or part of the product. Traditional multi-viewpoint calibration is mostly for camera combinations with overlapping fields of view, using a calibration plane for calibration. If the camera is facing at an angle that is too large, multi-viewpoint calibration will be performed by increasing the number of cameras. Traditional multi-viewpoint calibration is difficult to calibrate for camera combinations with non-overlapping fields of view, has low accuracy, and requires high hardware costs. It has poor versatility and cannot be deployed at low cost in industrial fields with multiple camera combinations with non-overlapping fields of view.

[0093] Based on this, the present application provides a camera calibration method, which is suitable for global calibration of cameras of different types and arbitrary postures, and especially solves the calibration problem of non-overlapping fields of view between cameras. The method uses a U-shaped calibration plate with multiple circular return light targets attached to the left and right ends, and locates different calibration plane feature points by shooting with the left and right cameras, and establishes a rotation and translation transformation relationship with the calibration plate coordinate system. The posture relationship between the left and right cameras can be solved according to the posture relationship between the left and right cameras and the calibration plate. The present invention enhances the applicability of multi-viewpoint calibration, improves calibration accuracy and reduces hardware costs.

[0094] Based on this, an embodiment of the present application provides a camera calibration method, which is applied to at least two cameras, wherein the at least two cameras include a first camera and a second camera, wherein the shooting ranges of the first camera and the second camera have no overlapping area, the first camera has a first coordinate system, and the second camera has a second coordinate system. Figure 1 The flowchart of the camera calibration method shown is as follows.

[0095] S100: Acquire a first image taken by a first camera for a preset first target and a second image taken by a second camera for a preset second target.

[0096] Among them, the first target and the second target are located on the same calibration plate, the calibration plate has a third coordinate system, the first target corresponds to the first calibration plate coordinates in the third coordinate system, and the second target corresponds to the second calibration plate coordinates in the third coordinate system, wherein the third coordinate system, the first calibration plate coordinates and the second calibration plate coordinates are determined.

[0097] It should be noted that during calibration, the positions of the first camera, the second camera and the calibration plate cannot be changed. Adjust the shooting direction of the first camera so that all first targets are within the shooting range of the first camera, and then adjust the internal parameters of the first camera, such as adjusting the focal length of the first camera, so that the first targets can be clearly presented in the first image taken by the first camera. Similarly, adjust the shooting direction of the second camera so that all second targets are within the shooting range of the second camera, and then adjust the internal parameters of the second camera, such as adjusting the focal length of the second camera, so that the second targets can be clearly presented in the second image taken by the second camera.

[0098] S102, determining first camera coordinates of a first target in a first coordinate system according to the first image; and determining second camera coordinates of a second target in a second coordinate system according to the second image.

[0099] Among them, the first image contains the image of the first target. Since the position of the first camera, the internal parameters, the position of the calibration plate and the position of the first target are fixed and known (even if the position of the first target relative to the first camera is changed multiple times to improve the calibration accuracy, the position relationship between the first target and the first camera is determined), the pixel coordinates of the first image can be determined, and then according to the position relationship between the first camera and the first target, the pixel coordinates can be converted into the first coordinate system to obtain the first camera coordinates. The second camera coordinates are the same and will not be repeated.

[0100] S104, obtaining a first pose relationship between the first camera and the calibration plate according to the first camera coordinates and the first calibration plate coordinates; obtaining a second pose relationship between the second camera and the calibration plate according to the second camera coordinates and the second calibration plate coordinates.

[0101] Among them, the first camera coordinates are the coordinates for the first target in the first coordinate system, and the first calibration plate coordinates are the coordinates for the first target in the third coordinate system. Since they are all coordinates for the first target, the first position relationship between the first camera and the calibration plate can be determined. For example, if the first camera coordinates of the first target are 0,0 and the first calibration plate coordinates are 1,0, it can be known that the first position relationship is that the first camera is located at a unit distance of the calibration plate in the x-axis direction. The second position relationship is the same and will not be repeated.

[0102] S106 . Obtain calibration results of the first camera and the second camera according to the first pose relationship and the second pose relationship.

[0103] The first pose relationship represents the position and direction relationship between the first camera and the calibration plate, and the second pose relationship represents the position and direction relationship between the second camera and the calibration plate. Since both are position and direction relationships with the calibration plate, the pose relationship between the first camera and the second camera, that is, the calibration result, can be determined.

[0104] By adopting this embodiment, a calibration plate including different side faces can be utilized, so that the first camera can shoot a first target on one side face and finally obtain the first pose relationship between the first camera and the calibration plate, and the second camera can shoot a second target on the other side face and finally obtain the second pose relationship between the second camera and the calibration plate. Since the first pose relationship and the second pose relationship have the same reference object, i.e., the calibration plate, the calibration result can finally be obtained based on the first pose relationship and the second pose relationship, thereby realizing the calibration of multiple cameras with non-overlapping shooting ranges.

[0105] In a possible embodiment of the present application, Figure 2 The flowchart for determining the first camera coordinates shown in the figure determines the first camera coordinates of the first target in the first coordinate system according to the first image, including:

[0106] S200: Determine first pixel coordinates of a first target according to first target information in a first image.

[0107] The first target information refers to information of pixels in the first image used to characterize the first target, such as pixel coordinates of pixels used to characterize the first target. It should be noted that pixel coordinates are coordinates in a pixel coordinate system, and the pixel coordinate system may be another coordinate system independent of the first coordinate system, the second coordinate system, and the third coordinate system (and the first image and the second image correspond to different pixel coordinate systems), and the pixel coordinate system of the first image may also be the first coordinate system, in which case the pixel coordinate system of the second image is the second coordinate system.

[0108] S202: Determine first camera coordinates according to first pixel coordinates and camera parameters of a first camera.

[0109] The camera parameter refers to the focal length, which may specifically include the vertical focal length and the horizontal focal length of the camera.

[0110] After knowing the first pixel coordinates and camera parameters, the first camera coordinates can be determined. Specifically, the origin position of the pixel coordinate system where the first pixel coordinates are located is known, and the origin position of the first coordinate system is also known, so the relationship between the two origin positions can be determined. In this way, the first pixel coordinates can be converted to the first coordinate system according to the relationship between the two origin positions, thereby obtaining the first camera coordinates. For example, the relationship between the two origin positions is that the distance in the direction of the x-axis differs by 1 unit, then the horizontal coordinate of the first pixel coordinate is subtracted by 1 or added by 1 to obtain the first camera coordinates. Whether to add 1 or subtract 1 is determined according to the positive direction of the x-axis, and will not be repeated.

[0111] According to this embodiment, the first pixel coordinates of the first target in the first image are determined and then combined with the camera parameters to obtain the first camera coordinates. Since the first pixel coordinates and the camera parameters are both related to the first camera, more accurate first camera coordinates can be obtained efficiently.

[0112] Optionally, in an implementation manner of this embodiment, the first target information includes a first edge point of the first target in the first image.

[0113] Specifically, the first image contains pixel points of the image representing the first target, and the pixel points used to enclose the outline of the first target among the pixel points of the image representing the first target are the first edge points. That is to say, in the first image, the pixel points that represent the outline of the image of the first target are the first edge points. It should be noted that when using the first edge point, at least the position of the first edge point in the first image, that is, the pixel coordinates of the first edge point, and the number of the first edge points, etc., need to be known. All of this information that needs to be used can be included in the first target information, and this embodiment does not specifically limit this. In addition, after the pixel point as the first edge point is determined, the corresponding pixel coordinates and the number of the first edge points can also be determined, which will not be repeated.

[0114] Determining a first pixel coordinate of the first target according to first target information in the first image includes:

[0115] The gradient direction and intensity of the first edge point corresponding to each first target are calculated to obtain the first initial center position of the corresponding first target.

[0116] The original meaning of gradient is a vector (vector), which means that the directional derivative of a function at this point reaches the maximum value along this direction, that is, the function changes fastest along this direction (the direction of this gradient, that is, the gradient direction) at this point, and the rate of change is the largest (the modulus of the gradient). Intensity refers to the pixel brightness of the first edge point.

[0117] Processing the first initial center position of each first target by using a least squares algorithm to obtain the first center position of each first target in the first image;

[0118] Taking the first center position as the first pixel coordinate;

[0119] Among them, the least squares algorithm includes:

[0120]

[0121] Among them, x0 is the horizontal coordinate of the first center position, t0 is the vertical coordinate of the first center position, r is the radius, x i is the x coordinate of the ith first edge point, y i is the y coordinate of the first edge point, and b is the number of the first edge points.

[0122] In this embodiment, the first initial center position is obtained by using the gradient direction and intensity of the first edge point, and then the first center position is calculated using the least squares algorithm. This is equivalent to performing a preprocessing first and then a fine processing. This method helps to improve the accuracy of the first pixel coordinates.

[0123] Optionally, in an implementation manner of this embodiment, in an optional implementation manner of the embodiment of the present application, determining the first camera coordinates according to the first pixel coordinates and a camera parameter of the first camera includes:

[0124] The first camera coordinates are calculated based on the following formula:

[0125] V i =(f x ×(p ix -c x ),f y ×(p iy -c y ),1);

[0126] Among them, V i is the normal vector of the i-th first target in the first coordinate system, f x is the horizontal focal length of the first camera, f y is the vertical focal length of the first camera, c x is the horizontal coordinate of the principal point of the first camera, c y is the vertical coordinate of the principal point of the first camera, Pix is the horizontal coordinate of the i-th first target in the first coordinate system, p iy is the vertical coordinate of the i-th first target in the first coordinate system, and i is a positive integer.

[0127] By adopting this embodiment, the first camera coordinates are obtained by calculating using a formula, which helps to improve the accuracy of the first camera coordinates and improve the calculation efficiency.

[0128] Optionally, in an implementation of this embodiment, in an optional implementation of the embodiment of the present application, obtaining a first pose relationship between the first camera and the calibration plate according to the first camera coordinates and the first calibration plate coordinates includes:

[0129] According to the first camera coordinates and the first calibration plate coordinates, a first position relationship is calculated using a first algorithm, wherein the first algorithm includes a SolvePnP algorithm, and a formula of the first algorithm includes:

[0130]

[0131] in, is the transformation matrix from the third coordinate system to the first coordinate system, SolvePnP is the calculation function, P is the coordinate set of all first targets in the third coordinate system, and V is the direction vector set of all first targets in the first coordinate system.

[0132] By adopting this implementation method, the first pose relationship is obtained by using a formula to calculate, which helps to improve the accuracy of the first pose relationship and improve the calculation efficiency.

[0133] Optionally, in an implementation of the present embodiment, in an optional implementation of the embodiment of the present application, determining the second camera coordinates of the second target in the second coordinate system based on the second image is the same as the method of determining the first camera coordinates of the first target in the first coordinate system based on the first image.

[0134] Specifically, determining the second camera coordinates of the second target in the second coordinate system according to the second image includes:

[0135] determining a second pixel coordinate of the second target according to the second target information in the second image;

[0136] The second camera coordinates are determined according to the second pixel coordinates and the camera parameters of the second camera.

[0137] By adopting this implementation method, the second pixel coordinates of the second target in the second image are determined, and then combined with the camera parameters to obtain the second camera coordinates. Since the second pixel coordinates and the camera parameters are both related to the second camera, more accurate second camera coordinates can be obtained efficiently.

[0138] Optionally, in an implementation manner of the present embodiment, in an optional implementation manner of the embodiment of the present application, the second target information includes a second edge point of the second target in the second image;

[0139] Determining a second pixel coordinate of the second target according to the second target information in the second image includes:

[0140] Calculate the gradient direction and intensity of the second edge point corresponding to each second target to obtain the second initial center position of the corresponding second target;

[0141] Processing the second initial center position of each second target by using a least squares algorithm to obtain a second center position of each second target in the second image;

[0142] Using the second center position as the second pixel coordinate;

[0143] Among them, the least squares algorithm includes:

[0144]

[0145] Among them, x0 is the horizontal coordinate of the second center position, y0 is the vertical coordinate of the second center position, r is the radius, x i is the x coordinate of the second edge point, y i is the y coordinate of the second edge point, and n is the number of the second edge points.

[0146] With this implementation, the second initial center position is obtained by first using the gradient direction and intensity of the second edge point, and then the second center position is calculated using the least squares algorithm, which is equivalent to performing a preprocessing first and then a fine processing. This method helps to improve the accuracy of the second pixel coordinates.

[0147] Optionally, in an implementation manner of this embodiment, in an optional implementation manner of the embodiment of the present application, determining the second camera coordinates according to the second pixel coordinates and the camera parameters of the second camera includes:

[0148] The pixel coordinates and the camera parameters of the second camera determine the second camera coordinates, including:

[0149] The second camera coordinates are calculated based on the following formula:

[0150] V i =(f x ×(p ix -c x ),f y ×(p iy -c y ),1);

[0151] Among them, V iis the normal vector of the i-th second target in the second coordinate system, f x is the horizontal focal length of the second camera, f y is the vertical focal length of the second camera, c x is the horizontal coordinate of the principal point of the second camera, c y is the vertical coordinate of the principal point of the second camera, p ix is the horizontal coordinate of the i-th second target in the second coordinate system, p iy is the vertical coordinate of the i-th second target in the second coordinate system, and i is a positive integer.

[0152] By adopting this implementation, the second camera coordinates are calculated using a formula, which helps to improve the accuracy of the second camera coordinates and improve the calculation efficiency.

[0153] Optionally, in an implementation of this embodiment, in an optional implementation of the embodiment of the present application, obtaining a second posture relationship between the second camera and the calibration plate according to the second camera coordinates and the second calibration plate coordinates includes:

[0154] According to the second camera coordinates and the second calibration plate coordinates, a second position relationship is calculated using a second algorithm, wherein the second algorithm includes a SolvePnP algorithm, and a formula of the second algorithm includes:

[0155]

[0156] in, is the transformation matrix from the third coordinate system to the second coordinate system, SolvePnP is the calculation function, P is the coordinate set of all second targets in the third coordinate system, and V is the direction vector set of all second targets in the second coordinate system.

[0157] By adopting this implementation method, the second posture relationship is calculated using a formula, which helps to improve the accuracy of the second posture relationship and improve calculation efficiency.

[0158] Optionally, in an implementation of this embodiment, the calibration plate is in a shape in which the length direction, the width direction and the height direction are perpendicular to each other.

[0159] That is, the length direction is perpendicular to both the width direction and the height direction, the width direction is perpendicular to both the length direction and the height direction, and the height direction is perpendicular to both the length direction and the width direction.

[0160] The method also includes:

[0161] The target position on the calibration plate is taken as the coordinate origin, and the length, width and height directions of the calibration plate are taken as the directions of each coordinate axis to obtain the third coordinate system.

[0162] Among them, the target position can be pre-set, that is, the target position is known, and at the same time, the length direction, width direction and height direction are also known, so the third coordinate system is also equivalent to being known and can be determined without calculation.

[0163] With this implementation, once the structure of the calibration plate is determined, the target position can be determined. At the same time, since the length, width and height directions can all be determined, the third coordinate system can be obtained. That is, when the calibration plate is determined, the third coordinate system can also be determined.

[0164] Optionally, in an implementation of this embodiment, the method further includes:

[0165] Determine the coordinates of the first calibration plate according to the distance between the first target and the target position in the length direction, the distance between the first target and the target position in the width direction, and the distance between the first target and the target position in the height direction;

[0166] The coordinates of the second calibration plate are determined according to the distance of the second target from the target position in the length direction, the distance of the second target from the target position in the width direction, and the distance of the second target from the target position in the height direction.

[0167] It should be noted that after the structure of the calibration plate is determined, the position of the first target on the calibration plate is also determined and will not change. Even if the position of the calibration plate changes, the first target remains stationary relative to the target position. Therefore, the coordinates of the first calibration plate can be directly obtained or determined. The second target is the same as the first target.

[0168] With this implementation, after the third coordinate system is determined, since the positions of the first target and the second target on the calibration plate are determined, the coordinates of the first calibration plate and the coordinates of the second calibration plate can be determined.

[0169] Optionally, in an implementation of this embodiment, the first target is located on a first side of the calibration plate, and the second target is located on a second side of the calibration plate;

[0170] The method also includes:

[0171] Determine the coordinates of the first calibration plate according to the positional relationship between the first side surface and the target position and the position of the first target on the first side surface;

[0172] The coordinates of the second calibration plate are determined according to the positional relationship between the second side surface and the target position and the position of the second target on the second side surface.

[0173] With this implementation, after the calibration plate is determined, the positional relationship between the first side surface, the second side surface and the target position can also be determined, and since the first target is on the first side surface, the first calibration plate coordinates can be obtained by the position of the first target on the first side surface. Similarly, the second calibration plate coordinates can be obtained.

[0174] Optionally, in an implementation of this embodiment, the first side surface and the second side surface are the same side surface of the calibration plate, and the first target and the second target are located in different areas of the same side surface.

[0175] Alternatively, the first side surface and the second side surface are different sides of the calibration plate, and a distance between the first side surface and the second side surface enables the first camera and the second camera to be simultaneously located between the first side surface and the second side surface.

[0176] For ease of understanding, when the first side and the second side are the same side of the calibration plate, the calibration plate as a whole is a straight plate or a curved plate, the left area of ​​the calibration plate is the first side, and the right area of ​​the calibration plate is the second side.

[0177] When the first side surface and the second side surface are different sides of the calibration plate, the calibration plate as a whole is in a V-shape, that is, an edge line of the first side surface and an edge line of the second side surface are collinear, and the first camera and the second camera are located in the calibration plate for photographing the first target and the second target. The calibration plate as a whole can also be in a 7-shape, and the first side surface and the second side surface are both vertical surfaces and are respectively connected to the horizontal surface.

[0178] With this implementation, when the first side surface and the second side surface are different sides, it is helpful to calibrate the first camera and the second camera with different shooting directions. When the first side surface and the second side surface are the same side surface, since the first target and the second target are in different areas, it is helpful to calibrate the first camera and the second camera with a spacing distance but the same shooting direction.

[0179] Optionally, in an implementation of the present embodiment, in an optional implementation of the embodiment of the present application, acquiring a first image taken by a first camera for a preset first target and a second image taken by a second camera for a preset second target includes:

[0180] The first image and the second image of the calibration plate are obtained at multiple positions.

[0181] In one embodiment, the first side surface where the first target is located can be rotated so that the posture relationship between the first target and the first camera changes, so that the first camera can shoot the first target in different postures to obtain multiple first images, and then process the first images respectively to calculate the first posture relationship corresponding to each first image. Similarly, the second posture relationship corresponding to each second image can be obtained. Then, the calibration result is obtained according to each first posture relationship and each second posture relationship.

[0182] By adopting this implementation method, the amount of data is increased by acquiring the first image and the second image when the calibration plate has multiple postures, which helps to improve the calculation accuracy of the first posture relationship and the second posture relationship, thereby improving the accuracy of the calibration result.

[0183] Optionally, in an implementation manner of the present embodiment, in an optional implementation manner of the embodiment of the present application, the shooting directions of the first camera and the second camera are opposite.

[0184] By adopting this implementation, the first camera and the second camera with opposite shooting directions can be calibrated, thereby improving the application scope of the camera calibration method provided in this application.

[0185] Optionally, in an implementation of the present embodiment, in an optional implementation of the embodiment of the present application, obtaining a calibration result of the first camera and the second camera according to the first pose relationship and the second pose relationship includes:

[0186] The first posture relationship and the second posture relationship are calculated using the following formula to obtain the calibration result;

[0187]

[0188] in, To calibrate the results, is the transformation matrix from the third coordinate system to the first coordinate system during the i-th shooting, is the transformation matrix from the third coordinate system to the second coordinate system during the i-th shooting, where i is a positive integer.

[0189] By adopting this implementation method, the calibration result is obtained by using the formula to calculate, which helps to improve the accuracy and calculation efficiency of the calibration result.

[0190] Optionally, in an implementation of the present embodiment, in an optional implementation of the embodiment of the present application, after obtaining the calibration results of the first camera and the second camera according to the first pose relationship and the second pose relationship, the method further includes:

[0191] The calibration result is processed by the second algorithm to obtain an optimization result, wherein the second algorithm includes an LM posture optimization algorithm, and the formula of the second algorithm includes:

[0192]

[0193] Where M is the number of shots, R is the rotation matrix from the second coordinate system to the first coordinate system, T is the translation matrix from the second coordinate system to the first coordinate system, and R i is the rotation matrix from the second coordinate system to the first coordinate system calculated during the i-th shooting, T i is the translation matrix from the second coordinate system to the first coordinate system calculated during the i-th shooting, E is the overall calculation error, and i is a positive integer.

[0194] By adopting this implementation method, the calibration results are optimized, which helps to improve the accuracy of the calibration results.

[0195] The present application embodiment provides a calibration board for calibrating at least two cameras, wherein the at least two cameras include a first camera and a second camera. Figure 3 As shown, the calibration plate 1 includes a first side wall 11, a second side wall 12 and a connecting portion 13 connecting the first side wall 11 and the second side wall 12, the first side wall 11 has a first side surface for setting a target, and the second side wall 12 has a second side surface for setting a target;

[0196] The first side wall 11 and the second side wall 12 are respectively located at two ends of the connecting portion 13 in the length direction;

[0197] At least one first target 2 is arranged on the first side surface for being photographed by the first camera 4;

[0198] At least one second target 3 is arranged on the second side surface for being photographed by the second camera 5;

[0199] In the third coordinate system based on the calibration plate 1 , the first calibration plate coordinates of the first target 2 and the second calibration plate coordinates of the second target 3 are determined.

[0200] By adopting this implementation method, a calibration plate with different side surfaces is used, so that the first camera can shoot the first target on the first side to obtain the posture relationship between the first camera and the calibration plate, and the second camera can shoot the second target on the second side to obtain the posture relationship between the second camera and the calibration plate. Since there is the same reference object, i.e., the calibration plate, the calibration result can be finally obtained based on the two posture relationships, thereby realizing the calibration of multiple cameras with non-overlapping shooting ranges.

[0201] Optionally, in an implementation of the present embodiment, in an optional implementation of the embodiment of the present application, the first side wall 11 and the second side wall 12 are both rotatably connected to the connecting portion 13 .

[0202] By adopting this implementation method, it is easy to adjust the posture between the first side surface and the first camera, and the posture between the second side surface and the second camera, thereby facilitating the acquisition of multiple sets of data and improving the calibration accuracy of the first camera and the second camera.

[0203] Optionally, in an implementation of this embodiment, the calibration plate 1 corresponds to the third coordinate system with the target position on the connecting portion 13 as the coordinate origin and the length direction, width direction and height direction of the calibration plate 1 as the directions of each coordinate axis;

[0204] The first calibration plate coordinates of the first target 2 in the third coordinate system can be determined according to the positional relationship between the first side surface and the target position and the position of the first target 2 on the first side surface;

[0205] The second calibration plate coordinates of the second target 3 in the third coordinate system can be determined according to the positional relationship between the second side surface and the target position and the position of the second target 3 on the second side surface.

[0206] Specifically, the position may include the angle between the first side surface and the connecting portion 13. Taking the first target 2 as an example, after knowing the angle, firstly, according to the position of the first target 2 on the first side surface (specifically, the distance of the first target from the corresponding side of the first side surface in the first horizontal direction and the distance of the first target from the corresponding side of the first side surface in the vertical direction), the distance between the first target and the target position in the first horizontal direction and the distance between the first target and the target position in the vertical direction can be known by using the angle to calculate the trigonometric function, and finally, according to the distance of the first target in the second horizontal direction of the first side surface, the distance between the first target and the target position in the second horizontal direction (specifically, equal) can be determined, thereby determining the coordinates of the first calibration plate.

[0207] The connecting portion 13 may be a side wall, such as a third side wall, or may be a connecting rod, which is not specifically limited in this embodiment.

[0208] Optionally, in one implementation of the present embodiment, the first side wall 11 and the second side wall 12 have the same size and shape, the connecting portion 13 fixes the distance between the first side wall 11 and the second side wall 12, the length direction of the connecting portion 13 is perpendicular to the first side surface 11 and the second side surface 12, and along the length direction of the connecting portion 13, the projection of the first side wall 11 on the second side wall 12 coincides with the second side wall 12.

[0209] That is to say, the calibration plate is n-shaped as a whole.

[0210] To facilitate understanding of the calibration process, the camera calibration system is illustrated as follows:

[0211] 1. If Figure 3 The figure shows the principle diagram of the multi-view calibration method under the opposite field of view of the proposed camera, which mainly consists of a U-shaped calibration plate, a group of cameras with opposite fields of view (including the first camera and the second camera), and multiple circular return light targets (including the first target and the second target):

[0212] (1) The U-shaped calibration plate is a three-sided calibration plate with known dimensions. The calibration surfaces on both sides are perpendicular to the middle connecting surface, and circular return light targets need to be installed on the calibration surfaces on both sides according to the installation position. The calibration plate coordinate system (that is, the third coordinate system) is established. The coordinate of the circular return light target on the left side in the calibration plate coordinate system is P L = {P 1L ,...,P NL}, the coordinates of the circular return light target on the right side in the calibration plate coordinate system are P R = {P 1R ,...,P NR}, where N>=4, is the number of circular return light targets.

[0213] (2) The left and right cameras (the left one is the first camera and the right one is the second camera) are a group of cameras to be calibrated from multiple viewpoints. Their fields of view are opposite or non-overlapping. The camera coordinate systems are established separately, with the origin being the optical center of the camera, the Z axis being the optical axis of the camera, the X axis being parallel to the horizontal axis of the camera imaging plane, and the Y axis being parallel to the vertical axis of the camera imaging plane. The relative position of the cameras is kept unchanged during the calibration process.

[0214] 2. If Figure 4 As shown in the figure, it is a flowchart of the implementation of the multi-viewpoint calibration method under the opposite field of view of the camera. The main process is as follows:

[0215] (1) Adjust the calibration plate posture, camera focal length, and aperture size so that the left and right cameras can clearly capture the circular return light targets on both sides. The grayscale contrast between the circular return light target and the calibration plate plane is obvious. The grayscale value of the circular return light target imaging is close to the upper limit of 255, and the background imaging is close to the lower limit of 0. Establish the calibration plate coordinate system (that is, the third coordinate system), determine the X, Y, and Z axis directions and the origin, and calculate P according to the placement of the circular return light target. L and P R ; Calibrate the left and right cameras to determine the camera intrinsic parameters and distortion parameters.

[0216] (2) Adjust the calibration plate posture multiple times and take photos. Use the circle positioning algorithm to detect the pixel coordinates of the circular return light target on the camera image plane to obtain the UV Li ={UV 1Li ,...,UV NLi} and UV Ri ={UV 1Ri ,...,UV NRi}, where i=1,...,M,M>=3 is the number of photos; the pixel coordinates UV of the circular return light target of the left camera are used Li , calibration plate coordinate system coordinate P L As well as the camera intrinsic distortion parameters, the pose relationship between the left camera and the calibration plate can be obtained according to the SolvePnP algorithm. Similarly, the position relationship between the right camera and the calibration plate can be obtained

[0217] (3) Based on the position relationship between multiple groups of cameras and calibration plates That is, the position relationship between the left and right cameras can be obtained as shown in formula (1):

[0218]

[0219] Using the algorithm, the final calibration result can be obtained As shown in formula (2):

[0220]

[0221] 3. If Figure 5 The schematic diagram of the monocular vision positioning method based on the circular target with reflected light is shown in the figure. The main processing steps are as follows:

[0222] (1) Establish the circular return light target in the calibration plate coordinate system coordinate P = {P1, ..., P N}, take a picture of it with a camera, dedistort the image according to the distortion parameters, and locate its pixel coordinates p = {p1,...,p N}.

[0223] (2) According to the camera internal parameters [f x ,f y ,c x ,c y ] Determine the direction of the circular return light target in the camera coordinate system V = {V1,...,V N}, where the ray direction of the i-th circular return light target is shown in formula (3):

[0224] V i =(f x ×(p ix -c x ),f y ×(p iy -c y ),1) (3)

[0225] (3) According to P = {P1, ..., P N} and V i =(f x ×(p ix-c x ),f y ×(p iy -c y ),1), use SolvePnP algorithm to directly solve the pose relationship between the camera and the calibration plate As shown in formula (4):

[0226]

[0227] 4. If Figure 6 The figure shows the flow chart of the sub-pixel circle positioning algorithm, and its main process is as follows:

[0228] (1) First, the input image is preprocessed and Gaussian filtering is used to reduce the impact of image noise; then the Canny edge detection algorithm is used to find the edge of the circle in the image.

[0229] (2) Calculate the gradient direction and intensity of the edge points of the circle in the image and initialize the center position of the circle.

[0230] (3) Use the least squares algorithm to determine the center position (x0, y0), as shown in formula (5):

[0231]

[0232] Where n is the number of edge points.

[0233] 5. If Figure 7 The figure shows the flow chart of the LM pose optimization algorithm. The main processing steps are as follows:

[0234] (1) For M Camera pose transformation relationship, each relationship consists of a rotation matrix R i and the translation matrix T i express.

[0235] (2) Use the average value to initialize the rotation matrix and translation matrix.

[0236] (3) Establish the minimum error function E as shown in formula (6):

[0237]

[0238] (4) Use the LM (Levenberg-Marquardt) algorithm to iteratively calculate the final camera pose relationship

[0239] In the above-mentioned embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments. The steps shown in the relevant flow chart can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flow chart, in some cases, the steps shown or described can be executed in an order different from that here. In other words, the order of steps described in the foregoing embodiments is only an example, and it is also within the scope of protection of the embodiments of the present application to reasonably adjust the order of steps based on the content of the embodiments of the present application.

[0240] The sequence of serial numbers or introduction of the embodiments of the present application is for description only and does not represent the superiority or inferiority of the embodiments.

[0241] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0242] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0243] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0244] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can access, or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a digital versatile disc (DVD)), or a semiconductor medium (e.g., a solid state disk (SSD)). It is worth noting that the computer-readable storage medium mentioned in the embodiment of the present application may be a non-volatile storage medium, in other words, a non-transient storage medium.

[0245] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data for analysis, stored data, displayed data, etc.) and signals involved in the embodiments of this application are all authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions. For example, the scene data of the current frame in the three-dimensional virtual scene, the device information of the client, and the scene interaction information involved in the embodiments of this application are all obtained with full authorization.

[0246] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A camera calibration method, characterized in that: Applied to at least two cameras, the at least two cameras include a first camera and a second camera, the shooting ranges of the first camera and the second camera have no overlapping area, the first camera has a first coordinate system, and the second camera has a second coordinate system, the method includes: Acquire a first image taken by the first camera for a preset first target and a second image taken by the second camera for a preset second target, wherein the first target and the second target are located on the same calibration plate, the calibration plate has a third coordinate system, the first target corresponds to a first calibration plate coordinate in the third coordinate system, and the second target corresponds to a second calibration plate coordinate in the third coordinate system, wherein the third coordinate system, the first calibration plate coordinate, and the second calibration plate coordinate are determined; determining a first camera coordinate of the first target in the first coordinate system according to the first image; determining a second camera coordinate of the second target in the second coordinate system according to the second image; Obtaining a first pose relationship between the first camera and the calibration plate according to the first camera coordinates and the first calibration plate coordinates; Obtaining a second position and posture relationship between the second camera and the calibration plate according to the second camera coordinates and the second calibration plate coordinates; A calibration result of the first camera and the second camera is obtained according to the first posture relationship and the second posture relationship.

2. The camera calibration method according to claim 1, characterized in that: The determining, according to the first image, a first camera coordinate of the first target in the first coordinate system comprises: Determining a first pixel coordinate of the first target according to first target information in the first image; The first camera coordinates are determined according to the first pixel coordinates and camera parameters of the first camera.

3. The camera calibration method according to claim 2, characterized in that: The first target information includes a first edge point of the first target in the first image; The determining the first pixel coordinates of the first target according to the first target information in the first image includes: Calculating the gradient direction and intensity of the first edge point corresponding to each of the first targets to obtain a first initial center position of the corresponding first target; Processing the first initial center position of each of the first targets using a least squares algorithm to obtain a first center position of each of the first targets in the first image; The first center position is used as the first pixel coordinate.

4. The camera calibration method according to claim 2, characterized in that: The determining the first camera coordinates according to the first pixel coordinates and the camera parameters of the first camera includes: The first camera coordinates are calculated based on the following formula: V i =(f x ×(p ix -c x ),f y ×(p iy -c y ),1); Among them, V i is the normal vector of the i-th first target in the first coordinate system, f x is the horizontal focal length of the first camera, f y is the vertical focal length of the first camera, c x is the horizontal coordinate of the image principal point of the first camera, c y is the vertical coordinate of the principal point of the first camera, p ix is the horizontal coordinate of the i-th first target in the first coordinate system, p iy is the vertical coordinate of the i-th first target in the first coordinate system, and i is a positive integer.

5. The camera calibration method according to claim 1, characterized in that: The method of determining the second camera coordinates of the second target in the second coordinate system according to the second image is the same as the method of determining the first camera coordinates of the first target in the first coordinate system according to the first image.

6. The camera calibration method according to any one of claims 1 to 5, characterized in that: The calibration plate is shaped such that the length direction, the width direction and the height direction are perpendicular to each other; The method further comprises: The third coordinate system is obtained by taking the target position on the calibration plate as the coordinate origin and taking the length direction, width direction and height direction of the calibration plate as the directions of each coordinate axis.

7. The camera calibration method according to claim 6, characterized in that: The method further comprises: Determine the coordinates of the first calibration plate according to the distance between the first target and the target position in the length direction, the distance between the first target and the target position in the width direction, and the distance between the first target and the target position in the height direction; The coordinates of the second calibration plate are determined according to the distance of the second target from the target position in the length direction, the distance of the second target from the target position in the width direction, and the distance of the second target from the target position in the height direction.

8. The camera calibration method according to claim 6, characterized in that: The first target is located on a first side of the calibration plate, and the second target is located on a second side of the calibration plate; The method further comprises: Determining the coordinates of a first calibration plate according to a positional relationship between the first side surface and the target position and a position of the first target on the first side surface; The coordinates of the second calibration plate are determined according to the positional relationship between the second side surface and the target position and the position of the second target on the second side surface.

9. The camera calibration method according to claim 8, characterized in that: The first side surface and the second side surface are the same side surface of the calibration plate, and the first target and the second target are located in different areas of the same side surface. Alternatively, the first side surface and the second side surface are different sides of the calibration plate, and a distance between the first side surface and the second side surface enables the first camera and the second camera to be simultaneously located between the first side surface and the second side surface.

10. The camera calibration method according to claim 1, characterized in that: The acquiring of a first image captured by the first camera for a preset first target and a second image captured by the second camera for a preset second target includes: The first image and the second image are acquired at multiple positions of the calibration plate.

11. The camera calibration method according to claim 1, characterized in that: The shooting directions of the first camera and the second camera are opposite.

12. The camera calibration method according to claim 1, characterized in that: The obtaining the calibration results of the first camera and the second camera according to the first posture relationship and the second posture relationship includes: The first posture relationship and the second posture relationship are calculated using the following formula to obtain the calibration result; in, To calibrate the results, is the transformation matrix from the third coordinate system to the first coordinate system during the i-th shooting, is the transformation matrix from the third coordinate system to the second coordinate system during the i-th shooting, where i is a positive integer.

13. The camera calibration method according to claim 1, characterized in that: After obtaining the calibration results of the first camera and the second camera according to the first posture relationship and the second posture relationship, the method further includes: The calibration result is processed by a second algorithm to obtain an optimization result, wherein the second algorithm includes an LM posture optimization algorithm, and the formula of the second algorithm includes: Where M is the number of shots, and its value range is a positive integer. R is the rotation matrix from the second coordinate system to the first coordinate system. T is the translation matrix from the second coordinate system to the first coordinate system. R i is the rotation matrix from the second coordinate system to the first coordinate system calculated during the i-th shooting, T i is the translation matrix from the second coordinate system to the first coordinate system calculated during the i-th shooting, E is the overall calculation error, and i is a positive integer.

14. A calibration plate, characterized in that: Used to calibrate at least two cameras, the at least two cameras include a first camera and a second camera, the calibration plate includes a first side wall, a second side wall and a connecting portion connecting the first side wall and the second side wall, the first side wall has a first side surface for setting a target, and the second side wall has a second side surface for setting a target; The first side wall and the second side wall are respectively located at two ends of the connecting portion in the length direction; At least one first target is disposed on the first side surface for being photographed by the first camera; At least one second target is disposed on the second side surface for being photographed by the second camera; In a third coordinate system based on the calibration plate, the first calibration plate coordinates of the first target and the second calibration plate coordinates of the second target are determined.

15. The calibration plate according to claim 14, characterized in that: The first side wall and the second side wall are both rotatably connected to the connecting portion.

16. The calibration plate according to claim 14, characterized in that: The calibration plate corresponds to the third coordinate system with the target position on the connecting portion as the coordinate origin and the length direction, width direction and height direction of the calibration plate as the directions of the respective coordinate axes; The first calibration plate coordinates of the first target in the third coordinate system can be determined according to the positional relationship between the first side surface and the target position and the position of the first target on the first side surface; The second calibration plate coordinates of the second target in the third coordinate system can be determined according to the positional relationship between the second side surface and the target position and the position of the second target on the second side surface.

17. The calibration plate according to claim 14, characterized in that: The first side wall and the second side wall have the same size and shape, the connecting portion fixes the distance between the first side wall and the second side wall, the length direction of the connecting portion is perpendicular to the first side surface and the second side surface, and along the length direction of the connecting portion, the projection of the first side wall on the second side wall coincides with the second side wall.

Images