A camera parameter calibration method, device and equipment

Abstract

This application provides a method, apparatus, and device for calibrating camera parameters. The method includes: determining the pixel coordinates of a calibration point in an image coordinate system based on a calibration two-dimensional image; determining the physical coordinates of the calibration point in each of multiple configured system coordinate systems; generating multiple coordinate point pairs corresponding to the system coordinate systems, each coordinate point pair including the pixel coordinates of the calibration point and the physical coordinates of the calibration point in the system coordinate system; determining the camera intrinsic and extrinsic parameters of the two-dimensional camera based on the multiple coordinate point pairs corresponding to the system coordinate systems, and determining the motion velocity of the measured object based on the camera extrinsic parameters; selecting a target system coordinate system from all system coordinate systems based on the motion velocity corresponding to each system coordinate system, and calibrating the camera intrinsic and extrinsic parameters of the two-dimensional camera corresponding to the target system coordinate system. The technical solution of this application simplifies calibration operations, reduces calibration time, and saves calibration manpower costs.

Description

Technical Field

[0001] This application relates to the field of vision technology, and in particular to a method, apparatus and device for calibrating camera parameters. Background Technology

[0002] In the field of vision-based multidimensional information measurement and recognition, multiple cameras can be used to acquire information about objects in different dimensions. Then, temporal correlation prediction technology is used to bind the recognition information from multiple cameras on the same object to obtain complete information about the object. For example, in the application of package volume measurement and optical symbol recognition in logistics systems, 3D cameras can be used for package volume measurement and classification, multiple small-field-of-view 2D cameras can be used for optical symbol recognition, and large-field-of-view 2D cameras can be used to obtain panoramic images of packages. Then, the 3D positions of multiple cameras are correlated, and temporal correlation prediction technology is combined to obtain the volume value, category information, optical symbol information, and panoramic image of each package. Based on this information, subsequent operations such as piece-rate billing, preventing missed scans, package verification, and classified loading can be performed.

[0003] To correlate the 3D positions of multiple cameras, it is necessary to calibrate the intrinsic and extrinsic parameters of each camera and correlate their 3D positions based on these parameters. However, there is no reasonable calibration method for each camera in the relevant technologies, resulting in problems such as cumbersome calibration operations, long calibration times, and high labor costs. Summary of the Invention

[0004] This application provides a method for calibrating camera parameters. A calibration object is placed on the measurement object, and the calibration object includes multiple calibration points. During the movement of the measurement object, the method includes:

[0005] A 2D image of the calibration object is acquired using a 2D camera. The pixel coordinates of the calibration point in the image coordinate system are determined based on the calibration image. The physical coordinates of the calibration point in each system coordinate system are determined based on the configured multiple system coordinate systems.

[0006] For each system coordinate system, multiple coordinate point pairs corresponding to the system coordinate system are generated. The coordinate point pairs include the pixel coordinates of the calibration point and the physical coordinates of the calibration point in the system coordinate system.

[0007] The camera intrinsic and extrinsic parameters of the two-dimensional camera are determined based on multiple coordinate point pairs corresponding to the system coordinate system, and the motion velocity of the measured object is determined based on the camera extrinsic parameters.

[0008] Based on the motion velocity corresponding to each system coordinate system, a target system coordinate system is selected from all system coordinate systems to calibrate the camera intrinsic and extrinsic parameters corresponding to the target system coordinate system for the two-dimensional camera.

[0009] This application provides a camera parameter calibration device. A calibration object is placed on the object being measured, and the calibration object includes multiple calibration points. During the movement of the object being measured, the device includes:

[0010] The acquisition module is used to acquire a calibration two-dimensional image of the measurement object using a two-dimensional camera, determine the pixel coordinates of the calibration point in the image coordinate system based on the calibration two-dimensional image, determine the physical coordinates of the calibration point in each system coordinate system based on multiple configured system coordinate systems, and generate multiple coordinate point pairs corresponding to each system coordinate system, wherein the coordinate point pairs include the pixel coordinates of the calibration point and the physical coordinates of the calibration point in the system coordinate system.

[0011] The determination module is used to determine the camera intrinsic parameters and camera extrinsic parameters of the two-dimensional camera based on multiple coordinate point pairs corresponding to the system coordinate system, and to determine the motion speed of the measured object based on the camera extrinsic parameters;

[0012] The calibration module is used to select the target system coordinate system from all system coordinate systems based on the motion velocity corresponding to the system coordinate system, and to calibrate the camera intrinsic and extrinsic parameters corresponding to the target system coordinate system for the 2D camera.

[0013] This application provides an electronic device, including: a processor and a machine-readable storage medium, wherein the machine-readable storage medium stores machine-executable instructions that can be executed by the processor; wherein the processor is used to execute the machine-executable instructions to implement the camera parameter calibration method described above in this application.

[0014] As can be seen from the above technical solutions, in this embodiment, only a calibration object (such as a calibration cloth or calibration plate) is needed to calibrate the camera's extrinsic and intrinsic parameters, as well as the motion speed, effectively reducing the manufacturing and carrying costs of the calibration object. By acquiring a calibration 2D image of the object to be measured using a 2D camera, and combining this with the vertical height between the 2D camera and the object (i.e., the mounting height of the 2D camera), the camera's intrinsic and extrinsic parameters can be calculated, and the 2D camera's intrinsic and extrinsic parameters can be calibrated, greatly improving implementation efficiency and saving labor costs. By associating each camera with the system coordinate system, while decoupling the cameras from each other, the scalability of camera calibration is improved. By configuring an initial focal length value for the camera, and based on the initial focal length value, the target parameter values ​​corresponding to the parameters to be calibrated by the 2D camera are determined based on multiple coordinate point pairs, thereby reducing the amount of calculation, saving computing resources, and improving calibration speed. Based on the above camera calibration method, calibration operations can be simplified, calibration time reduced, and calibration labor costs saved. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments of this application or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained based on these drawings of the embodiments of this application.

[0016] Figure 1 This is a flowchart illustrating a camera parameter calibration method according to one embodiment of this application.

[0017] Figure 2 This is a schematic diagram of an application scenario in one embodiment of this application;

[0018] Figure 3 This is a schematic diagram of the calibration process of a 3D camera in one embodiment of this application;

[0019] Figure 4 This is a schematic diagram of the calibration process of a two-dimensional camera in one embodiment of this application;

[0020] Figure 5 This is a flowchart illustrating a camera parameter calibration method according to one embodiment of this application.

[0021] Figure 6 This is a schematic diagram of the camera parameter calibration device in one embodiment of this application;

[0022] Figure 7 This is a hardware structure diagram of an electronic device according to one embodiment of this application. Detailed Implementation

[0023] The terminology used in the embodiments of this application is for the purpose of describing particular embodiments only and is not intended to limit the application. The singular forms “a,” “the,” and “the” as used in this application and claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to any and all possible combinations comprising one or more of the associated listed items.

[0024] It should be understood that although the terms first, second, third, etc., may be used to describe various information in embodiments of this application, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" may also be interpreted as "when," "when," or "in response to a determination."

[0025] This application proposes a method for calibrating camera parameters. A calibration object (which can be a calibration cloth or a calibration plate) is placed on the object being measured. The calibration object includes multiple calibration points. During the movement of the object being measured, see... Figure 1 The diagram shown is a flowchart of the method, which may include:

[0026] Step 101: Acquire a calibration two-dimensional image of the object to be measured using a two-dimensional camera, and determine the pixel coordinates of the calibration point in the image coordinate system based on the calibration two-dimensional image; determine the physical coordinates of the calibration point in each of the multiple configured system coordinate systems based on the configured system coordinate systems.

[0027] Step 102: For each system coordinate system, generate multiple coordinate point pairs corresponding to that system coordinate system (multiple coordinate point pairs corresponding to multiple calibration points). For each coordinate point pair corresponding to a calibration point, the coordinate point pair includes the pixel coordinates of the calibration point and the physical coordinates of the calibration point in that system coordinate system.

[0028] Step 103: For each system coordinate system, determine the camera intrinsic and extrinsic parameters of the two-dimensional camera based on multiple coordinate point pairs corresponding to the system coordinate system, and determine the motion speed of the measured object based on the camera extrinsic parameters, and record the motion speed of the measured object as the motion speed corresponding to the system coordinate system.

[0029] For example, determining the intrinsic and extrinsic parameters of a 2D camera based on multiple coordinate point pairs corresponding to the system coordinate system may include, but is not limited to: configuring an initial focal length value for the camera, and determining the target parameter value corresponding to the parameter to be calibrated of the 2D camera based on the initial focal length value and multiple coordinate point pairs. Based on the target parameter value corresponding to the parameter to be calibrated, the vertical height between the 2D camera and the measurement object, and the initial focal length value, the target focal length value corresponding to the 2D camera's focal length is determined; based on the target focal length value corresponding to the camera's focal length and the target parameter value corresponding to the parameter to be calibrated, the intrinsic and extrinsic parameters of the 2D camera are determined.

[0030] For example, based on the target parameter value corresponding to the parameter to be calibrated, the vertical height between the 2D camera and the measured object, and the initial focal length value, the target focal length value corresponding to the 2D camera's focal length is determined. This can include, but is not limited to, determining a candidate focal length value corresponding to the 2D camera's focal length based on the target parameter value, the vertical height between the 2D camera and the measured object, and the initial focal length value. If the difference between the candidate focal length value and the initial focal length value is less than a preset threshold, the candidate focal length value can be determined as the target focal length value corresponding to the camera's focal length; otherwise, if the difference between the candidate focal length value and the initial focal length value is not less than a preset threshold, the candidate focal length value is determined as the initial focal length value, and the operation of determining the target parameter value corresponding to the parameter to be calibrated of the 2D camera based on the initial focal length value and multiple coordinate point pairs is returned.

[0031] For example, the parameters to be calibrated may include, but are not limited to: camera center, distortion coefficient, rotation parameters, and translation parameters, and the translation parameters include X-axis translation parameters, Y-axis translation parameters, and Z-axis translation parameters; based on this, based on the target parameter values, the vertical height between the 2D camera and the measurement object, and the initial focal length value, the candidate focal length value corresponding to the camera focal length of the 2D camera is determined, which may include, but is not limited to: determining an adjustment factor based on the target parameter values ​​corresponding to the vertical height and the Z-axis translation parameters, and adjusting the initial focal length value based on the adjustment factor to obtain the candidate focal length value corresponding to the camera focal length of the 2D camera.

[0032] For example, the parameters to be calibrated may include, but are not limited to, the camera center, distortion coefficients, rotation parameters, and translation parameters. Based on the initial focal length value and multiple coordinate point pairs, the target parameter values ​​corresponding to the calibrated parameters of the 2D camera are determined. This may include, but is not limited to: configuring an initial center value for the camera center and initial coefficient values ​​for the distortion coefficients; and determining the initial values ​​of the rotation parameters and translation parameters based on the initial focal length value, initial center value, initial coefficient values, and multiple coordinate point pairs. The initial center value, initial coefficient values, initial values ​​of the rotation parameters, and initial values ​​of the translation parameters are then optimized (e.g., through nonlinear optimization) to obtain the target center value corresponding to the camera center, the target coefficient value corresponding to the distortion coefficients, the target rotation parameter value corresponding to the rotation parameters, and the target translation parameter value corresponding to the translation parameters.

[0033] For example, target parameter values ​​include, but are not limited to, the target center value corresponding to the camera center, the target coefficient value corresponding to the distortion coefficient, the target value of the rotation parameter corresponding to the rotation parameter, the target value of the translation parameter corresponding to the translation parameter, the target focal length value corresponding to the camera focal length, the target center value corresponding to the camera center, and the target coefficient value corresponding to the distortion coefficient; these are the intrinsic parameters of the 2D camera. The target values ​​of the rotation parameter corresponding to the rotation parameter and the target values ​​of the translation parameter corresponding to the translation parameter are the extrinsic parameters of the 2D camera.

[0034] For example, the calibration two-dimensional image may include a first calibration two-dimensional image and a second calibration two-dimensional image during the motion of the measured object. Determining the motion speed of the measured object based on the camera extrinsic parameters may include, but is not limited to: if the camera extrinsic parameters corresponding to the first calibration two-dimensional image include a first X-axis translation parameter value and a first Y-axis translation parameter value, and the camera extrinsic parameters corresponding to the second calibration two-dimensional image include a second X-axis translation parameter value and a second Y-axis translation parameter value, then the distance change can be determined based on the first X-axis translation parameter value, the first Y-axis translation parameter value, the second X-axis translation parameter value, and the second Y-axis translation parameter value. Based on the distance change and a target duration, the motion speed of the measured object can be determined, where the target duration can be the time interval between the first calibration two-dimensional image and the second calibration two-dimensional image.

[0035] Step 104: Select the target system coordinate system from all system coordinate systems based on the motion velocity corresponding to each system coordinate system, and calibrate the camera intrinsic and extrinsic parameters corresponding to the target system coordinate system for the two-dimensional camera.

[0036] For example, selecting a target system coordinate system from all system coordinate systems based on the motion velocity corresponding to each system coordinate system may include, but is not limited to: selecting a target motion velocity corresponding to all cameras based on the motion velocity corresponding to each system coordinate system of the 2D camera and the motion velocity corresponding to each system coordinate system of the other cameras (the other cameras may include 2D cameras and / or 3D cameras), and determining the system coordinate system corresponding to the target motion velocity as the target system coordinate system.

[0037] In one possible implementation, a calibration 3D point cloud of the measurement object can be acquired using a 3D camera, and the 3D spatial coordinates corresponding to the calibration points can be determined based on the calibration 3D point cloud. Based on multiple configured system coordinate systems, the physical coordinates corresponding to the calibration points in each system coordinate system are determined. For each system coordinate system, multiple coordinate point pairs corresponding to that system coordinate system are generated (multiple coordinate point pairs corresponding to multiple calibration points). For each coordinate point pair corresponding to a calibration point, the coordinate point pair may include the 3D spatial coordinates of the calibration point and the physical coordinates corresponding to the calibration point in that system coordinate system. For each system coordinate system, based on the calibrated camera intrinsic parameters of the 3D camera and the multiple coordinate point pairs corresponding to that system coordinate system, the camera extrinsic parameters of the 3D camera can be determined, and the motion velocity of the measurement object can be determined based on these camera extrinsic parameters. Based on the motion velocity corresponding to each system coordinate system, a target system coordinate system is selected from all system coordinate systems to calibrate the camera extrinsic parameters corresponding to the target system coordinate system for the 3D camera.

[0038] In one possible implementation, when a target object is placed on the measurement object and the measurement object is moving at a constant linear speed, a two-dimensional image of the target object can be acquired using a two-dimensional camera, and a three-dimensional point cloud of the target object can be acquired using a three-dimensional camera. The three-dimensional spatial coordinates of the target object can be determined based on the target three-dimensional point cloud, and these three-dimensional spatial coordinates can be converted into the initial physical coordinates of the target object in the system coordinate system based on the camera intrinsic and extrinsic parameters of the three-dimensional camera. Based on the measurement object's motion speed, a preset duration, and the initial physical coordinates, the target physical coordinates of the target object in the system coordinate system are determined. Then, the target physical coordinates can be converted into target pixel coordinates of the target object in the image coordinate system based on the camera intrinsic and extrinsic parameters of the two-dimensional camera, and the pixel position corresponding to the target pixel coordinates can be located from the target two-dimensional image corresponding to the preset duration.

[0039] As can be seen from the above technical solutions, in this embodiment, only a calibration object (such as a calibration cloth or calibration plate) is needed to calibrate the camera's extrinsic and intrinsic parameters, as well as the motion speed, effectively reducing the manufacturing and carrying costs of the calibration object. By acquiring a calibration 2D image of the object to be measured using a 2D camera, and combining this with the vertical height between the 2D camera and the object (i.e., the mounting height of the 2D camera), the camera's intrinsic and extrinsic parameters can be calculated, and the 2D camera's intrinsic and extrinsic parameters can be calibrated, greatly improving implementation efficiency and saving labor costs. By associating each camera with the system coordinate system, while decoupling the cameras from each other, the scalability of camera calibration is improved. By configuring an initial focal length value for the camera, and based on the initial focal length value, the target parameter values ​​corresponding to the parameters to be calibrated by the 2D camera are determined based on multiple coordinate point pairs, thereby reducing the amount of calculation, saving computing resources, and improving calibration speed. Based on the above camera calibration method, calibration operations can be simplified, calibration time reduced, and calibration labor costs saved.

[0040] The technical solutions described above in the embodiments of this application will be explained below in conjunction with specific application scenarios.

[0041] In the field of vision-based multidimensional information measurement and recognition, multiple cameras can be used to acquire information about objects in different dimensions. Then, temporal correlation prediction technology is used to bind the recognition information from multiple cameras on the same object to obtain complete information about the object. For example, in the application of package volume measurement and optical symbol recognition in logistics systems, 3D cameras can be used for package volume measurement and classification, multiple small-field-of-view 2D cameras can be used for optical symbol recognition, and large-field-of-view 2D cameras can be used to obtain panoramic images of packages. Then, the 3D positions of multiple cameras are correlated, and temporal correlation prediction technology is combined to obtain the volume value, category information, optical symbol information, and panoramic image of each package. Based on this information, subsequent operations such as piece-rate billing, preventing missed scans, package verification, and classified loading can be performed.

[0042] To correlate the 3D positions of multiple cameras, it is necessary to calibrate the intrinsic and extrinsic parameters of each camera and correlate their 3D positions based on these parameters. However, there is no reasonable calibration method for each camera in the relevant technologies, resulting in problems such as cumbersome calibration operations, long calibration times, and high labor costs.

[0043] To address the above findings, this embodiment proposes a multi-camera calibration and motion velocity calibration method. This method requires laying a calibration cloth or calibration plate on the object being measured, allowing the object to move at a constant speed. The mounting height of each 2D camera (i.e., the vertical distance between the 2D camera and the object being measured) is measured and stored in the 2D camera. During the uniform motion of the object, each 2D camera acquires multiple frames of calibration 2D images. Once the object's path covers the measurement areas of all cameras, the intrinsic and extrinsic parameters of the 2D cameras can be calibrated based on the calibration 2D images and the camera mounting heights, and the motion velocity of the object can be calibrated. In this method, the calibration material is a calibration cloth (calibration plate) that can be printed on-site, and the cloth (plate) needs to be laid manually on the object being measured. Based on this, the two-dimensional camera only needs to know the installation height and acquire multiple frames of calibration two-dimensional images of the object being measured to complete the internal and external parameter calibration of the two-dimensional camera. This effectively reduces the cost of manufacturing and carrying the calibration object, greatly improves the implementation efficiency, simplifies the calibration operation of camera parameters, reduces calibration time, and saves calibration manpower costs.

[0044] In one possible implementation, see Figure 2 The diagram shown is an application scenario illustration of an embodiment of this application. A calibration cloth or calibration plate is laid on the measurement object. The following description will use the calibration cloth as an example.

[0045] Because it is necessary to perform temporal correlation and prediction of objects on the measurement object and to correlate information from multiple cameras, the measurement object can be an object moving in a straight line at a constant speed, such as a logistics conveyor belt moving in a straight line at a constant speed, a cross belt moving in a straight line at a constant speed, a robot moving in a straight line at a constant speed, or a vehicle moving in a straight line at a constant speed. There is no restriction on the type of measurement object, and the measurement object can also be called a measurement system.

[0046] Calibration cloth can be laid on the object being measured. The calibration cloth can be printed and spliced ​​in sections, and there are no restrictions on its material; it can simply be laid statically on the object. For example, if the object being measured is a conveyor belt or cross-belt, the calibration cloth can be laid on top of it. If the object is a robot or vehicle, the calibration cloth can be laid on top of it. In the direction of movement of the object, it is only necessary to ensure that each camera can capture at least multiple frames of the complete calibration plate. These multiple frames are used to adjust the camera's extrinsic parameters.

[0047] The calibration cloth can be a calibration cloth with calibration grids, which can be a checkerboard grid or a dotted grid; there are no restrictions on the type of calibration grid. Figure 2In this example, we can use a chessboard as an example. Multiple calibration points on the calibration cloth can be determined based on the calibration grid. For instance, for a chessboard calibration grid, the intersection of the black and white chessboard grids can be used as the calibration point. For a dot calibration grid, the center of the circle can be used as the calibration point. Of course, the above are just examples; as long as the calibration points can be determined from the calibration cloth, it is acceptable.

[0048] See Figure 2 As shown, the example uses one 3D camera and three 2D cameras. The three 2D cameras can include two 2D barcode readers and one 2D panoramic camera. The 2D barcode readers are used to read the barcode information or other code information on the object, and the 2D panoramic camera is used to capture panoramic images of the object.

[0049] During the installation of the 2D camera, there are no restrictions on the installation angle or height; the 2D camera can be installed arbitrarily. Similarly, during the installation of the 3D camera, there are no restrictions on the installation angle or height; the 3D camera can be installed arbitrarily. After the 2D camera is installed, it is necessary to measure the vertical height between the 2D camera and the object being measured (i.e., the mounting height of the 2D camera). This vertical height can be the height of the 2D camera's optical center, which is perpendicular to the object being measured. Figure 2 In the diagram, the vertical distance between one 2D barcode reader camera and the object being measured is h1, the vertical distance between another 2D barcode reader camera and the object being measured is h2, and the vertical distance between the 2D panoramic camera and the object being measured is h3.

[0050] For both 2D and 3D cameras, camera intrinsic parameters can include camera focal length (fx, fy), camera center (cx, cy), and distortion coefficients (k1, k2, p1, p2, k3). For 2D cameras, the camera intrinsic parameters are used to transform a point Pc(x, y, z) in the camera coordinate system to a point Pi(x, y) in the image coordinate system, as shown in formula (1), which represents the transformation relationship between the camera coordinate system and the image coordinate system. If the calibration accuracy of the 2D camera is not strict, the camera intrinsic parameters of the 2D camera can be simplified, such as fy = fx, p1 = 0, p2 = 0, k3 = 0. In this case, see formula (2), which represents the transformation relationship between the camera coordinate system and the image coordinate system.

[0051] Pi(x,y)=[fx,fy,cx,cy][k1,k2,p1,p2,k3]Pc(x,y,z) Formula (1)

[0052] Pi(x,y)=[f,f,cx,cy][k1,k2]Pc(x,y,z) Formula (2)

[0053] As can be seen from formula (1), a point Pc(x, y, z) in the camera coordinate system can be converted to a point Pi(x, y) in the image coordinate system based on camera intrinsic parameters such as camera focal length (fx, fy), camera center (cx, cy), and distortion coefficients (k1, k2, p1, p2, k3). Similarly, a point Pi(x, y) in the image coordinate system can be converted to a point Pc(x, y, z) in the camera coordinate system based on the same camera intrinsic parameters. Obviously, based on points Pi(x, y) in the image coordinate system and points Pc(x, y, z) in the camera coordinate system, i.e., multiple coordinate point pairs, camera intrinsic parameters such as camera focal length (fx, fy), camera center (cx, cy), and distortion coefficients (k1, k2, p1, p2, k3) can also be determined.

[0054] As can be seen from formula (2), a point Pc(x, y, z) in the camera coordinate system can be converted to a point Pi(x, y) in the image coordinate system based on camera intrinsic parameters such as camera focal length f, camera center (cx, cy), and distortion coefficients (k1, k2). Similarly, a point Pi(x, y) in the image coordinate system can be converted to a point Pc(x, y, z) in the camera coordinate system based on camera intrinsic parameters such as camera focal length f, camera center (cx, cy), and distortion coefficients (k1, k2). Based on points Pi(x, y) in the image coordinate system and points Pc(x, y, z) in the camera coordinate system, i.e., multiple coordinate point pairs, camera intrinsic parameters such as camera focal length f, camera center (cx, cy), and distortion coefficients (k1, k2) can also be determined.

[0055] For both 2D and 3D cameras, camera extrinsic parameters can include rotation parameters R (Rx, Ry, Rz) and translation parameters T (Tx, Ty, Tz). Rx is the X-axis rotation parameter, Ry is the Y-axis rotation parameter, Rz is the Z-axis rotation parameter, Tx is the X-axis translation parameter, Ty is the Y-axis translation parameter, and Tz is the Z-axis translation parameter. The camera extrinsic parameters are used to transform a point Ps(x, y, z) in the system coordinate system to a point Pc(x, y, z) in the camera coordinate system, as shown in formula (3), which represents the transformation relationship between the system coordinate system and the camera coordinate system.

[0056] Pc(x,y,z)=[R|T]Ps(x,y,0) Formula (3)

[0057] As can be seen from formula (3), a point Ps(x, y, z) in the system coordinate system can be converted to a point Pc(x, y, z) in the camera coordinate system based on camera extrinsic parameters such as rotation parameter R and translation parameter T. Conversely, a point Pc(x, y, z) in the camera coordinate system can be converted to a point Ps(x, y, z) in the system coordinate system based on camera extrinsic parameters such as rotation parameter R and translation parameter T. Based on multiple coordinate point pairs, such as points Ps(x, y, z) in the system coordinate system and points Pc(x, y, z) in the camera coordinate system, the camera extrinsic parameters such as rotation parameter R and translation parameter T can also be determined.

[0058] For the calibration process of a 3D camera, see [link to documentation]. Figure 3 As shown, the following steps may be included:

[0059] Step 301: Acquire a calibration 3D point cloud of the measurement object using a 3D camera, and determine the 3D spatial coordinates of the calibration points (which can be multiple calibration points) based on the calibration 3D point cloud.

[0060] For example, a calibration cloth with calibration grids is laid on the object being measured, covering the measurement area of ​​the 3D camera and ensuring the camera can see the calibration grids. During the object's movement (e.g., uniform motion), the 3D camera acquires multiple sets of calibration 3D point clouds (at least two sets are sufficient), including 3D points specific to the object. Since the 3D camera can see the calibration grids, multiple calibration points can be determined based on them, such as the intersection of black and white checkerboard grids. Because there are many calibration points, K points can be selected from all calibration points. The value of K is configured empirically, representing the calibration operation based on K sets of coordinate pairs.

[0061] In summary, after obtaining the calibrated 3D point cloud, which includes the 3D spatial coordinates of all 3D points, the 3D spatial coordinates of K calibration points can be determined based on the calibrated 3D point cloud.

[0062] Step 302: Based on the configured multiple system coordinate systems, determine the physical coordinates of the calibration point in each system coordinate system. For example, for each of the K calibration points, determine the physical coordinates of the calibration point in each system coordinate system based on the configured multiple system coordinate systems.

[0063] For example, multiple system coordinate systems can be pre-configured. These system coordinate systems, also known as world coordinate systems or physical coordinate systems, are coordinate systems specific to the object being measured. A location on the calibration cloth can be used as the origin of the system coordinate system. Once the origin of the system coordinate system is known, the physical coordinates of the calibration point within that system coordinate system can be determined. For instance, a system coordinate system S1 can be configured from top to bottom and from left to right. The origin of system coordinate system S1 is the upper left corner of the calibration cloth, with the positive X-axis direction from left to right and the positive Y-axis direction from top to bottom. A system coordinate system S2 can be configured from right to left and from top to bottom. The origin of system coordinate system S2 is the upper right corner of the calibration cloth, with the positive X-axis direction from right to left and the positive Y-axis direction from top to bottom. Configure a system coordinate system S3 from bottom to top and from right to left. The origin of system coordinate system S3 is the lower right corner of the calibration cloth, with the positive X-axis direction from right to left and the positive Y-axis direction from bottom to top. Configure a system coordinate system S4 from left to right and from bottom to top. The origin of system coordinate system S4 is the lower left corner of the calibration cloth, with the positive X-axis direction from left to right and the positive Y-axis direction from bottom to top.

[0064] Based on the system coordinate system S1, for each of the K calibration points, the positional relationship between the calibration point and the origin of the system coordinate system S1 is known. For example, if the calibration point is separated from the origin by 4 chessboard squares and is located to the right of the origin, the physical coordinates of the calibration point in the system coordinate system S1 can be determined based on the positional relationship between the calibration point and the origin.

[0065] In summary, the physical coordinates of the K calibration points in system coordinate system S1 can be obtained. Similarly, the physical coordinates of the K calibration points in system coordinate system S2, the K calibration points in system coordinate system S3, and the K calibration points in system coordinate system S4 can be obtained.

[0066] Step 303: For each system coordinate system, generate multiple coordinate point pairs corresponding to that system coordinate system (multiple coordinate point pairs corresponding to multiple calibration points). For each coordinate point pair corresponding to a calibration point, the coordinate point pair includes the three-dimensional spatial coordinates of the calibration point and the physical coordinates of the calibration point in the system coordinate system.

[0067] For example, for each of the K calibration points, in step 301, the three-dimensional spatial coordinates corresponding to the calibration point are obtained, and in step 302, the physical coordinates of the calibration point in the system coordinate system S1 are obtained, thus obtaining the coordinate point pair corresponding to the calibration point. The coordinate point pair includes the three-dimensional spatial coordinates and the physical coordinates. In this way, K coordinate point pairs corresponding to the K calibration points can be obtained, that is, K coordinate point pairs corresponding to the system coordinate system S1. Similarly, K coordinate point pairs corresponding to the system coordinate system S2, the system coordinate system S3, and the system coordinate system S4 can be obtained.

[0068] Step 304: For each system coordinate system, based on the calibrated intrinsic parameters of the 3D camera and multiple coordinate point pairs corresponding to the system coordinate system, determine the target parameter values ​​corresponding to the extrinsic parameters of the 3D camera.

[0069] For example, when a 3D camera leaves the factory, its intrinsic parameters, such as focal length (fx, fy), camera center (cx, cy), and distortion coefficients (k1, k2, p1, p2, k3), are pre-calibrated. Since the intrinsic parameters are already pre-calibrated, it's only necessary to calibrate the extrinsic parameters. This includes calibrating the target values ​​for the rotation parameters R (Rx, Ry, Rz), such as the target values ​​for the X-axis, Y-axis, and Z-axis rotation parameters. Similarly, it involves calibrating the target values ​​for the translation parameters T (Tx, Ty, Tz), such as the target values ​​for the X-axis, Y-axis, and Z-axis translation parameters.

[0070] For example, by combining formulas (1) and (3), we can obtain the transformation relationship shown in formula (4), which is the transformation relationship between three-dimensional spatial coordinates and system coordinate system.

[0071] Pi(x,y,z)=[fx,fy,cx,cy][k1,k2,p1,p2,k3][R|T]Ps(x,y,z) Formula (4)

[0072] In formula (4), the camera focal length (fx, fy), camera center (cx, cy) and distortion coefficients (k1, k2, p1, p2, k3) are camera intrinsic parameters, which are known values. The rotation parameter R and translation parameter T are camera extrinsic parameters, which are unknown values ​​and are parameter values ​​that need to be calibrated. The coordinate point pairs can include three-dimensional spatial coordinates (corresponding to Pi) and physical coordinates in the system coordinate system (corresponding to Ps). After substituting K coordinate point pairs into formula (4), the target values ​​of the rotation parameter R and the translation parameter T can be obtained, and then the target values ​​of the rotation parameter and translation parameter of the three-dimensional camera can be determined.

[0073] In summary, by substituting the K coordinate pairs of system coordinate system S1 into formula (4), the camera extrinsic parameters corresponding to system coordinate system S1 can be obtained. Similarly, by substituting the K coordinate pairs of system coordinate system S2 into formula (4), the camera extrinsic parameters corresponding to system coordinate system S2 can be obtained, and so on.

[0074] Step 305: Determine the motion velocity of the object being measured based on the camera extrinsic parameters corresponding to each system coordinate system, and record the motion velocity of the object being measured as the motion velocity corresponding to the system coordinate system.

[0075] For example, at acquisition time 1, a calibration 3D point cloud P1 for the measured object is acquired using a 3D camera. At acquisition time 2, a calibration 3D point cloud P2 for the measured object is acquired using the 3D camera. That is, the calibration 3D point cloud can include both the calibration 3D point cloud P1 and the calibration 3D point cloud P2 during the movement of the measured object. When performing steps 301-304 based on the calibration 3D point cloud P1, the camera extrinsic parameters corresponding to the calibration 3D point cloud P1 can be obtained. The target value of the translation parameter corresponding to the X-axis translation parameter in the camera extrinsic parameters is recorded as the first X-axis translation parameter value, and the target value of the translation parameter corresponding to the Y-axis translation parameter in the camera extrinsic parameters is recorded as the first Y-axis translation parameter value. When performing steps 301-304 based on the calibration 3D point cloud P2, the camera extrinsic parameters corresponding to the calibration 3D point cloud P2 can be obtained. The target value of the translation parameter corresponding to the X-axis translation parameter in the camera extrinsic parameters is recorded as the second X-axis translation parameter value, and the target value of the translation parameter corresponding to the Y-axis translation parameter in the camera extrinsic parameters is recorded as the second Y-axis translation parameter value. Based on this, the distance change can be determined based on the first X-axis translation parameter value, the first Y-axis translation parameter value, the second X-axis translation parameter value, and the second Y-axis translation parameter value. The movement speed of the measured object can be determined based on the distance change and the target duration. The target duration can be the time interval between acquisition time 1 and acquisition time 2.

[0076] For example, the speed of the object being measured can be calculated using the following formula: Of course, this formula is just an example and is not a limitation. In this formula, v(x, y) represents the speed of the measured object, Δt represents the target duration, and Δ(tx, ty) represents the change in distance. The first X-axis translation parameter value and the first Y-axis translation parameter value can be mapped to one position, and the second X-axis translation parameter value and the second Y-axis translation parameter value can be mapped to another position. The change in distance is the distance between these two positions.

[0077] In summary, based on the camera extrinsic parameters corresponding to system coordinate system S1, the motion velocity V1 of the measured object can be determined; based on the camera extrinsic parameters corresponding to system coordinate system S2, the motion velocity V2 of the measured object can be determined; based on the camera extrinsic parameters corresponding to system coordinate system S3, the motion velocity V3 of the measured object can be determined; and based on the camera extrinsic parameters corresponding to system coordinate system S4, the motion velocity V4 of the measured object can be determined.

[0078] Step 306: Select the target system coordinate system from all system coordinate systems based on the motion velocity corresponding to each system coordinate system, calibrate the camera extrinsic parameters corresponding to the target system coordinate system for the 3D camera, and calibrate the motion velocity corresponding to the target system coordinate system for the 3D camera to complete the calibration work.

[0079] For example, based on the motion velocity corresponding to each system coordinate system of the 3D camera and the motion velocity corresponding to each system coordinate system of the other cameras besides the 3D camera, a target motion velocity corresponding to all cameras is selected, and the system coordinate system corresponding to the target motion velocity is determined as the target system coordinate system.

[0080] For example, suppose there is a 3D camera, a 2D camera 1, and a 2D camera 2. The motion velocity V1 corresponding to the system coordinate system S1 of the 3D camera is the same as the motion velocity corresponding to the system coordinate system S3 of the 2D camera 1, and the motion velocity V1 corresponding to the system coordinate system S1 of the 3D camera is the same as the motion velocity corresponding to the system coordinate system S4 of the 2D camera 2. Then, the system coordinate system S1 of the 3D camera can be used as the target system coordinate system, and the camera extrinsic parameters corresponding to the system coordinate system S1 of the 3D camera can be calibrated.

[0081] For the calibration process of a 2D camera, see [link to documentation]. Figure 4 As shown, the following steps may be included:

[0082] Step 401: Acquire a calibration two-dimensional image of the object being measured using a two-dimensional camera, and determine the pixel coordinates of the calibration points (which can be multiple calibration points) in the image coordinate system based on the calibration two-dimensional image. The calibration two-dimensional image can be an RGB image or a grayscale image.

[0083] For example, a calibration cloth with calibration grids is laid on the object being measured, covering the measurement area of ​​the 2D camera, and the camera can see the calibration grids. During the movement of the object (e.g., uniform motion), the 2D camera acquires multiple frames of calibration 2D images of the object (at least two frames are sufficient). These calibration 2D images include pixel coordinates of the object. Since the 2D camera can see the calibration grids, multiple calibration points can be determined based on them, such as the intersection of black and white checkerboard grids. Because there are many calibration points, M calibration points can be selected from all calibration points. The value of M is configured empirically, representing the calibration operation based on M sets of coordinate pairs.

[0084] In summary, after obtaining the calibration two-dimensional image, the pixel coordinates of each calibration point in the image coordinate system can be determined based on the calibration two-dimensional image, that is, the pixel coordinates of M calibration points in the image coordinate system are obtained.

[0085] Step 402: Based on the configured multiple system coordinate systems, determine the physical coordinates of the calibration point in each system coordinate system. For example, for each of the M calibration points, determine the physical coordinates of the calibration point in each system coordinate system based on the configured multiple system coordinate systems.

[0086] For example, multiple system coordinate systems can be pre-configured. A certain position of the calibration cloth can be used as the origin of the system coordinate system. After knowing the origin of the system coordinate system, the physical coordinates of the calibration point in the system coordinate system can be obtained. For example, a system coordinate system S1 can be configured from top to bottom and from left to right, with the origin at the upper left corner of the calibration cloth. A system coordinate system S2 can be configured from right to left and from top to bottom, with the origin at the upper right corner of the calibration cloth. A system coordinate system S3 can be configured from bottom to top and from right to left, with the origin at the lower right corner of the calibration cloth. A system coordinate system S4 can be configured from left to right and from bottom to top, with the origin at the lower left corner of the calibration cloth.

[0087] Based on the system coordinate system S1, for each of the M calibration points, the positional relationship between the calibration point and the origin of the system coordinate system S1 is known. Based on the positional relationship between the calibration point and the origin, the physical coordinates of the calibration point in the system coordinate system S1 can be determined.

[0088] In summary, the physical coordinates of the M calibration points in system coordinate system S1 can be obtained. Similarly, the physical coordinates of the M calibration points in system coordinate system S2, the M calibration points in system coordinate system S3, and the M calibration points in system coordinate system S4 can be obtained.

[0089] Step 403: For each system coordinate system, generate multiple coordinate point pairs corresponding to that system coordinate system (multiple coordinate point pairs corresponding to multiple calibration points). For each coordinate point pair corresponding to a calibration point, the coordinate point pair includes the pixel coordinates of the calibration point and the physical coordinates of the calibration point in that system coordinate system.

[0090] For example, for each of the M calibration points, in step 401, the pixel coordinates of the calibration point are obtained, and in step 402, the physical coordinates of the calibration point in the system coordinate system S1 are obtained, thus obtaining the coordinate point pair corresponding to the calibration point. The coordinate point pair includes the pixel coordinates and the physical coordinates. In this way, M coordinate point pairs corresponding to the M calibration points can be obtained, that is, M coordinate point pairs corresponding to the system coordinate system S1. Similarly, M coordinate point pairs corresponding to the system coordinate system S2, the system coordinate system S3, and the system coordinate system S4 can be obtained.

[0091] Step 404: For each system coordinate system (hereinafter, we will take one system coordinate system as an example), configure an initial focal length value (i.e., a fixed focal length value) for the camera focal length, and determine the target parameter value corresponding to the calibration parameters of the two-dimensional camera based on the initial focal length value and multiple coordinate points corresponding to the system coordinate system.

[0092] For example, when a 2D camera leaves the factory, its intrinsic and extrinsic parameters are not calibrated. Therefore, the intrinsic and extrinsic parameters outside the camera's focal length are used as parameters to be calibrated. These parameters include the camera center (cx, cy), distortion coefficients (k1, k2, p1, p2, k3), rotation parameters R (Rx, Ry, Rz), and translation parameters T (Tx, Ty, Tz). Based on this, the target center value corresponding to the camera center is determined (e.g., the target center value corresponding to camera center cx and camera center cy), and the target coefficient values ​​corresponding to the distortion coefficients are determined (e.g., the target coefficient value corresponding to distortion coefficient k1 and distortion coefficient k2). The target coefficient values ​​corresponding to the distortion coefficients p1, p2, and k3 are determined. The target values ​​of the rotation parameters R (Rx, Ry, Rz) are determined (e.g., the target values ​​of the rotation parameters corresponding to the X-axis, Y-axis, and Z-axis rotation parameters). The target values ​​of the translation parameters T (Tx, Ty, Tz) are determined (e.g., the target values ​​of the translation parameters corresponding to the X-axis, Y-axis, and Z-axis translation parameters).

[0093] In one possible implementation, by combining formulas (1) and (3), the transformation relationship shown in formula (5) can be obtained. Formula (5) is the transformation relationship between the image coordinate system and the system coordinate system.

[0094] Pi(x,y)=[fx,fy,cx,cy][k1,k2,p1,p2,k3][R|T]Ps(x,y,0) Formula (5)

[0095] In formula (5), the camera focal length (fx, fy) is the initial focal length value, which is configured according to experience, such as fx = fy = 1000. Of course, the initial focal length value can also be other values, and there is no restriction on this. By configuring the camera focal length (fx, fy) as the initial focal length value, the amount of calculation in the calibration process is reduced, the computing resources are saved, and the calibration speed is improved.

[0096] For example, the camera center (cx, cy) and distortion coefficients (k1, k2, p1, p2, k3) are camera intrinsic parameters, which are unknown values. The rotation parameter R and translation parameter T are camera extrinsic parameters, which are also unknown values. The above parameters are the parameters to be calibrated. The coordinate point pairs can include pixel coordinates (corresponding to Pi) in the image coordinate system and physical coordinates (corresponding to Ps) in the system coordinate system. Obviously, after substituting the M coordinate point pairs into formula (5), we can obtain the target center value corresponding to the camera center (cx, cy), the target coefficient value corresponding to the distortion coefficients (k1, k2, p1, p2, k3), the rotation parameter target value corresponding to the rotation parameter R (Rx, Ry, Rz), and the translation parameter target value corresponding to the translation parameter T (Tx, Ty, Tz).

[0097] In another possible implementation, in addition to configuring an initial focal length value for the camera, an initial center value can also be configured for the camera center, and initial coefficient values ​​can be configured for the distortion coefficients. For example, the center of the calibrated 2D image can be used as the initial center value, that is, half of the horizontal resolution of the calibrated 2D image can be used as the initial center value of the camera center cx, and half of the vertical resolution of the calibrated 2D image can be used as the initial center value of the camera center cy. The initial coefficient values ​​of the distortion coefficients can be configured to 0, such as k1=0, k2=0, p1=0, p2=0, k3=0. In this case, by combining formulas (1) and (3), the transformation relationship shown in formula (6) can be obtained, which is the transformation relationship between the image coordinate system and the system coordinate system. Since the initial coefficient value of the distortion coefficients is 0, formula (6) removes the distortion coefficients compared to formula (5).

[0098] Pi(x,y)=[fx,fy,cx,cy][R|T]Ps(x,y,0) Formula (6)

[0099] In formula (6), the camera focal length (fx, fy) is the initial focal length value, which can be configured based on experience, such as fx = fy = 1000. The camera center (cx, cy) is the center of the calibrated two-dimensional image, which is a known value. The distortion coefficients (k1, k2, p1, p2, k3) are 0, which is a known value. The rotation parameter R and the translation parameter T are camera extrinsic parameters, which are unknown values. The coordinate point pairs can include the pixel coordinates in the image coordinate system (corresponding to Pi) and the physical coordinates in the system coordinate system (corresponding to Ps). Obviously, after substituting the M coordinate point pairs into formula (6), the initial values ​​of the rotation parameters R (Rx, Ry, Rz) and the initial values ​​of the translation parameters T (Tx, Ty, Tz) can be obtained. That is, the initial values ​​of the rotation parameters and the initial values ​​of the translation parameters are determined based on the initial focal length value, the initial center value, the initial coefficient value, and multiple coordinate point pairs.

[0100] Then, the initial center value, initial coefficient value, initial value of rotation parameter, and initial value of translation parameter can be optimized (e.g., nonlinear optimization) to obtain the target center value corresponding to the camera center (cx, cy), the target coefficient value corresponding to the distortion coefficient (k1, k2, p1, p2, k3), the target value of the rotation parameter R (Rx, Ry, Rz) corresponding to the rotation parameter, and the target value of the translation parameter T (Tx, Ty, Tz) corresponding to the translation parameter.

[0101] For example, based on the least squares error of formula (5), nonlinear optimization methods (such as the LM method, which stands for Levenberg-Marquardt method and is a method for least squares estimation of regression parameters in nonlinear regression) can be used to jointly optimize the camera center (cx, cy), distortion coefficients (k1, k2, p1, p2, k3), rotation parameters R (Rx, Ry, Rz) and translation parameters T (Tx, Ty, Tz) to obtain the target center value corresponding to the camera center (cx, cy), the target coefficient value corresponding to the distortion coefficients (k1, k2, p1, p2, k3), the target value of the rotation parameter corresponding to the rotation parameter R (Rx, Ry, Rz), and the target value of the translation parameter corresponding to the translation parameter T (Tx, Ty, Tz). There are no restrictions on this optimization process.

[0102] For example, regarding the computational feasibility of formula (6), since three points can determine a plane, and each point can provide two equations (x, y) as in formula (6), six parameters can be calculated to meet the calculation requirements of camera extrinsic parameters (R|T). cx, cy, k1, and k2 are mainly related to distortion, and their calculation accuracy depends on whether the calibration grid covers enough camera field of view. They are less related to the number of images. In this embodiment, the timing diagram of the calibration cloth (calibration board) can cover the complete field of view of the camera in one direction. Therefore, these parameters can be determined by the timing diagram. In other words, the calibration of camera parameters can be completed.

[0103] In summary, based on the initial focal length of the camera, we can obtain the target center value corresponding to the camera center, the target coefficient value corresponding to the distortion coefficient, the target rotation parameter value corresponding to the rotation parameter R (Rx, Ry, Rz), and the target translation parameter value corresponding to the translation parameter T (Tx, Ty, Tz).

[0104] For example, the rotation parameters R corresponding to multiple calibrated 2D images can be the same, that is, the Rx corresponding to multiple calibrated 2D images is the same, the Ry corresponding to multiple calibrated 2D images is the same, and the Rz corresponding to multiple calibrated 2D images is the same. Of course, the rotation parameters R corresponding to multiple calibrated 2D images can also be different.

[0105] Step 405: Based on the target parameter values, the vertical height between the 2D camera and the measurement object, and the initial focal length value, determine the candidate focal length values ​​corresponding to the camera focal length (fx, fy) of the 2D camera.

[0106] For example, the parameters to be calibrated may include the camera center, distortion coefficients, rotation parameters, and translation parameters. The translation parameters include X-axis translation parameters, Y-axis translation parameters, and Z-axis translation parameters. That is, the target value of the translation parameter corresponding to the Z-axis translation parameter can be obtained. Based on this, an adjustment factor can be determined based on the vertical height and the target value of the translation parameter corresponding to the Z-axis translation parameter. The initial focal length value is then adjusted based on the adjustment factor to obtain the candidate focal length value corresponding to the camera focal length. For example, the candidate focal length value can be determined using formula (7). Of course, formula (7) is just an example, and this determination method is not limited.

[0107]

[0108] In formula (7), f represents the initial focal length value, that is, the camera focal length fx and the camera focal length fy both correspond to the initial focal length value, h represents the vertical height between the two-dimensional camera and the measured object, and Tz represents the target value of the translation parameter corresponding to the Z-axis translation parameter. f′ is used to represent the adjustment factor, and f′ is used to represent the candidate focal length value, that is, the camera focal length fx and the camera focal length fy both correspond to the candidate focal length value.

[0109] For example, based on the principle of triangulation, it is known that Tz in the inverse matrix of camera extrinsic parameters (R|T) has a nearly linear relationship with the camera focal length f. Tz is the height of the camera center perpendicular to the plane of the object being measured. Therefore, the initial value of the camera focal length f (i.e., the initial focal length value) can be arbitrarily set. After calculating the target center value corresponding to the camera center (cx, cy), the target coefficient value corresponding to the distortion coefficient (k1, k2, p1, p2, k3), the target value of the rotation parameter R (Rx, Ry, Rz), and the target value of the translation parameter T (Tx, Ty, Tz) according to formula (5) or formula (6), Tz in the inverse matrix of camera extrinsic parameters (R|T) can be obtained. Then, the initial focal length value is corrected according to formula (7) to obtain the candidate focal length value. The correct camera intrinsic parameters and camera extrinsic parameters are obtained through iterative updates.

[0110] Step 406: Determine whether the difference between the candidate focal length value and the initial focal length value is less than a preset threshold.

[0111] If yes, then proceed to step 407; otherwise, proceed to step 408.

[0112] For example, after obtaining the candidate focal length value, the difference between the candidate focal length value and the initial focal length value can be calculated, and it can be determined whether the difference between the candidate focal length value and the initial focal length value is less than a preset threshold. The preset threshold can be configured based on experience, and there is no restriction on the value of the preset threshold.

[0113] Step 407: Determine the candidate focal length value as the target focal length value corresponding to the camera focal length.

[0114] In summary, we can obtain the target focal length value corresponding to the camera focal length (fx, fy), the target center value corresponding to the camera center (cx, cy), the target coefficient value corresponding to the distortion coefficients (k1, k2, p1, p2, k3), the target rotation parameter value corresponding to the rotation parameter R (Rx, Ry, Rz), and the target translation parameter value corresponding to the translation parameter T (Tx, Ty, Tz). In this way, we can calibrate the intrinsic and extrinsic parameters of the 2D camera.

[0115] Step 408: Determine the candidate focal length value as the initial focal length value corresponding to the camera focal length, and return to step 404. Based on the updated initial focal length value and multiple coordinate points, redetermine the target parameter values ​​corresponding to the calibration parameters of the 2D camera. That is, the target parameter values ​​will be updated, thereby updating the camera focal length (fx, fy), camera center (cx, cy), distortion coefficients (k1, k2, p1, p2, k3), rotation parameters R (Rx, Ry, Rz), and translation parameters T through iterative updates until the correct camera intrinsic and extrinsic parameters are obtained, and calibrate the camera intrinsic and extrinsic parameters for the 2D camera.

[0116] Step 409: Determine the intrinsic and extrinsic parameters of the 2D camera based on the target focal length value corresponding to the camera focal length and the target parameter values ​​corresponding to the parameters to be calibrated. For example, the target parameter values ​​may include, but are not limited to, the target center value corresponding to the camera center, the target coefficient value corresponding to the distortion coefficient, the target rotation parameter value corresponding to the rotation parameter, and the target translation parameter value corresponding to the translation parameter. Based on this, the target focal length value corresponding to the camera focal length, the target center value corresponding to the camera center, and the target coefficient value corresponding to the distortion coefficient can be considered the intrinsic parameters of the 2D camera. Furthermore, the target rotation parameter value corresponding to the rotation parameter and the target translation parameter value corresponding to the translation parameter can be considered the extrinsic parameters of the 2D camera. Thus, the intrinsic and extrinsic parameters of the 2D camera are obtained; that is, for each system coordinate system, the intrinsic and extrinsic parameters of the 2D camera are determined based on multiple coordinate point pairs corresponding to that system coordinate system.

[0117] Step 410: Determine the motion speed of the object being measured based on the camera extrinsic parameters corresponding to each system coordinate system, and record the motion speed of the object being measured as the motion speed corresponding to the system coordinate system.

[0118] For example, at acquisition time 1, a calibration 2D image T1 for the measured object is acquired using a 2D camera. At acquisition time 2, a calibration 2D image T2 for the measured object is acquired using a 2D camera. That is, the calibration 2D images can include both calibration 2D images T1 and T2 during the movement of the measured object. When performing steps 401-409 based on the calibration 2D image T1, the camera extrinsic parameters corresponding to calibration 2D image T1 can be obtained. The target value of the translation parameter corresponding to the X-axis translation parameter in the camera extrinsic parameters is recorded as the first X-axis translation parameter value, and the target value of the translation parameter corresponding to the Y-axis translation parameter in the camera extrinsic parameters is recorded as the first Y-axis translation parameter value. When performing steps 401-409 based on the calibration 2D image T2, the camera extrinsic parameters corresponding to calibration 2D image T2 can be obtained. The target value of the translation parameter corresponding to the X-axis translation parameter in the camera extrinsic parameters is recorded as the second X-axis translation parameter value, and the target value of the translation parameter corresponding to the Y-axis translation parameter in the camera extrinsic parameters is recorded as the second Y-axis translation parameter value. Based on this, the distance change can be determined based on the first X-axis translation parameter value, the first Y-axis translation parameter value, the second X-axis translation parameter value, and the second Y-axis translation parameter value. The movement speed of the measured object can be determined based on the distance change and the target duration. The target duration can be the time interval between acquisition time 1 and acquisition time 2.

[0119] For example, the speed of the object being measured can be calculated using the following formula: Of course, this formula is just an example and is not a limitation. In this formula, v(x, y) represents the speed of the measured object, Δt represents the target duration, and Δ(tx, ty) represents the change in distance. The first X-axis translation parameter value and the first Y-axis translation parameter value can be mapped to one position, and the second X-axis translation parameter value and the second Y-axis translation parameter value can be mapped to another position. The change in distance is the distance between these two positions.

[0120] For example, after obtaining the camera extrinsic parameters [R|T], we can obtain the inverse matrix of the camera extrinsic parameters [R|T], decompose it to obtain the translation vector sequence [tx, ty, tz], and then obtain the target translation parameter value tx corresponding to the X-axis translation parameter and the target translation parameter value ty corresponding to the Y-axis translation parameter, thereby calculating the motion velocity v(x,y).

[0121] Since each frame of the calibration 2D image corresponds to a set of camera extrinsic parameters, the calibration 2D image T1 corresponds to a set of camera extrinsic parameters, and the calibration 2D image T2 corresponds to a set of camera extrinsic parameters. Based on the camera extrinsic parameters corresponding to the calibration 2D image T1 and the camera extrinsic parameters corresponding to the calibration 2D image T2, the motion velocity v(x,y) can be calculated.

[0122] In summary, based on the camera extrinsic parameters corresponding to system coordinate system S1, the motion velocity V1 of the measured object can be determined; based on the camera extrinsic parameters corresponding to system coordinate system S2, the motion velocity V2 of the measured object can be determined; based on the camera extrinsic parameters corresponding to system coordinate system S3, the motion velocity V3 of the measured object can be determined; and based on the camera extrinsic parameters corresponding to system coordinate system S4, the motion velocity V4 of the measured object can be determined.

[0123] Step 411: Select the target system coordinate system from all system coordinate systems based on the motion velocity corresponding to each system coordinate system. For example, based on the motion velocity corresponding to each system coordinate system of the 2D camera and the motion velocity corresponding to each system coordinate system of the other cameras besides the 2D camera, select the target motion velocity corresponding to all cameras, and determine the system coordinate system corresponding to the target motion velocity as the target system coordinate system.

[0124] Assuming there are a 3D camera, a 2D camera 1 (this 2D camera), and a 2D camera 2, the motion velocity corresponding to the system coordinate system S3 of the 2D camera 1 is the same as the motion velocity corresponding to the system coordinate system S1 of the 3D camera, and the motion velocity corresponding to the system coordinate system S3 of the 2D camera 1 is the same as the motion velocity corresponding to the system coordinate system S4 of the 2D camera 2. Then, the system coordinate system S3 of the 2D camera 1 is taken as the target system coordinate system.

[0125] In summary, it can be seen that the extrinsic parameters of all cameras can be aligned based on motion speed. For example, each camera corresponds to four system coordinate systems (a system coordinate system from top to bottom and left to right, a system coordinate system from right to left and top to bottom, a system coordinate system from bottom to top and right to left, and a system coordinate system from left to right and bottom to top). After obtaining the camera intrinsic and extrinsic parameters of all cameras in each system coordinate system, the motion speed is calculated based on the timestamp (i.e., the target duration). By selecting the extrinsic parameters of cameras with the same motion speed, the extrinsic parameters of all cameras can be aligned, thus completing the alignment of the extrinsic parameters of all cameras.

[0126] Step 412: Calibrate the camera intrinsic and extrinsic parameters corresponding to the target system coordinate system for the two-dimensional camera, and calibrate the motion velocity corresponding to the target system coordinate system for the two-dimensional camera to complete the calibration work.

[0127] In summary, the camera calibration process can be completed by calibrating the intrinsic and extrinsic parameters of a 2D camera and the motion velocity of the measured object, and by calibrating the extrinsic parameters of a 3D camera and the motion velocity of the measured object. After calibration, related applications can be executed based on the intrinsic and extrinsic parameters and motion velocity of the 2D and 3D cameras. For example, in the application of package volume measurement and optical symbol recognition in logistics systems, a 3D camera can be used for package volume measurement and classification, multiple small-field-of-view 2D cameras can be used for optical symbol recognition, and a large-field-of-view 2D camera can be used to obtain panoramic images of packages. Based on the intrinsic and extrinsic parameters of the 2D and 3D cameras, the 3D positions of multiple cameras can be correlated, and temporal correlation prediction technology can be used to obtain the volume value, category information, optical symbol information, and panoramic image of each package. This information is then used for subsequent operations such as piece-rate billing, preventing missed scans, package verification, and sorting and loading.

[0128] In one possible implementation, a target object is placed on the measurement object, and during the process of the measurement object moving at a constant speed in a straight line, a two-dimensional image of the target object can be acquired by a two-dimensional camera, and a three-dimensional point cloud of the target object can be acquired by a three-dimensional camera.

[0129] Then, the three-dimensional spatial coordinates corresponding to the target object are determined based on the target three-dimensional point cloud, and the three-dimensional spatial coordinates are converted into the initial physical coordinates of the target object in the system coordinate system based on the camera intrinsic and extrinsic parameters of the three-dimensional camera. For example, refer to the conversion relationship shown in formula (4). Formula (4) is the conversion relationship between three-dimensional spatial coordinates and the system coordinate system. In formula (4), the camera intrinsic and extrinsic parameters are known values. After substituting the three-dimensional spatial coordinates corresponding to the target object into formula (4), the initial physical coordinates of the target object in the system coordinate system can be obtained. For example, the three-dimensional camera can identify the three-dimensional spatial coordinates Pc(x, y, z) of the target object. Formula (4) can be transformed to obtain the conversion relationship shown in formula (8). In formula (8), only the camera extrinsic parameters of the three-dimensional camera are involved, and the camera intrinsic parameters of the three-dimensional camera are not involved. In this way, the three-dimensional spatial coordinates Pc(x, y, z) of the target object can be converted into the initial physical coordinates Ps(x, y, z) of the target object in the system coordinate system based on formula (8).

[0130] Ps(x, y, z) = [R|T] -1 Pc(x, y, z) Formula (8)

[0131] After obtaining the initial physical coordinates of the target object in the system coordinate system, the physical coordinates of the target object after a preset time period (denoted as the target physical coordinates) can be predicted. That is, the target object will move to the position corresponding to the target physical coordinates after the preset time period (such as after an interval of 10 seconds). For example, the target physical coordinates of the target object in the system coordinate system can be determined based on the motion speed of the measured object, the preset time period, and the initial physical coordinates. For example, the target physical coordinates of the target object in the system coordinate system can be determined using formula (9). In formula (9), Ps(x, y) represents the initial physical coordinates of the target object in the system coordinate system, that is, z in the initial physical coordinates Ps(x, y, z) is set to 0. Ps′(x, y) represents the target physical coordinates of the target object in the system coordinate system. s represents the motion speed of the measured object, that is, the pre-calibrated motion speed of the measured object. t represents the preset time period, which can be configured according to requirements.

[0132] Ps′(x,y)=s·t+Ps(x,y) Formula (9)

[0133] After obtaining the target object's physical coordinates in the system coordinate system, the target pixel coordinates of the target object after a preset time period can be predicted based on the physical coordinates. This allows for the association of the target pixel coordinates with the physical coordinates, thus enabling the prediction of the target object. Furthermore, the target pixel coordinates and their corresponding 3D spatial coordinates can also be associated; that is, the target pixel coordinates in the target 2D image can be associated with the target object's 3D spatial coordinates in the target 3D point cloud.

[0134] To predict the target pixel coordinates of a target object after a preset time interval based on its physical coordinates, the physical coordinates can be converted into the target pixel coordinates of the target object in the image coordinate system based on the camera intrinsic and extrinsic parameters of the 2D camera. For example, refer to the conversion relationship shown in formula (5), which is the conversion relationship between the image coordinate system and the system coordinate system. In formula (5), both the camera intrinsic and extrinsic parameters are known values. After substituting the physical coordinates of the target object into formula (5), the target pixel coordinates of the target object in the image coordinate system can be obtained. Based on this, the pixel position corresponding to the target pixel coordinates can be located from the target 2D image (i.e., the 2D image after the preset time interval) corresponding to the preset time interval. This pixel position is the position of the target object in the image coordinate system. The 2D camera can recognize the image information of the object. During information binding, the 3D information of the object in the system coordinate system is passively projected into the image based on the camera extrinsic and camera intrinsic parameters, and then the information binding is performed.

[0135] In one possible implementation, while a target object is placed on the measurement object and the measurement object is moving at a constant speed in a straight line, a two-dimensional image of the target object can be acquired using a two-dimensional camera, and a three-dimensional point cloud of the target object can be acquired using a three-dimensional camera. Then, the three-dimensional spatial coordinates corresponding to the target object are determined based on the target three-dimensional point cloud, and the three-dimensional spatial coordinates are converted into the target physical coordinates of the target object in the system coordinate system based on the camera intrinsic and extrinsic parameters of the three-dimensional camera.

[0136] After obtaining the target object's physical coordinates in the system coordinate system, the target pixel coordinates in the target 2D image can be determined based on these physical coordinates, thus associating the pixel coordinates with the physical coordinates. Furthermore, the target pixel coordinates can be associated with the corresponding 3D spatial coordinates of the physical coordinates; that is, the target pixel coordinates in the 2D image can be associated with the 3D spatial coordinates of the target object in the 3D point cloud. To determine the target pixel coordinates based on the physical coordinates, these physical coordinates can be converted into the target pixel coordinates in the image coordinate system based on the camera's intrinsic and extrinsic parameters.

[0137] As can be seen from the above technical solutions, in this embodiment, only a calibration object (such as a calibration cloth or calibration plate) is needed to calibrate the camera's extrinsic and intrinsic parameters, as well as the motion speed, effectively reducing the manufacturing and carrying costs of the calibration object. By acquiring a calibration 2D image of the object to be measured using a 2D camera, and combining this with the vertical height between the 2D camera and the object (i.e., the mounting height of the 2D camera), the camera's intrinsic and extrinsic parameters can be calculated, and the 2D camera's intrinsic and extrinsic parameters can be calibrated, greatly improving implementation efficiency and saving labor costs. By associating each camera with the system coordinate system, while decoupling the cameras from each other, the scalability of camera calibration is improved. By configuring an initial focal length value for the camera, and based on the initial focal length value, the target parameter values ​​corresponding to the parameters to be calibrated by the 2D camera are determined based on multiple coordinate point pairs, thereby reducing the amount of calculation, saving computing resources, and improving calibration speed. Based on the above camera calibration method, calibration operations can be simplified, calibration time reduced, and calibration labor costs saved.

[0138] The system coordinate system is determined by laying a calibration cloth on the plane of the measured object. During the movement of the measured object, the camera identifies the movement of the measured object to calculate the motion velocity. Each camera acquires images, and complete time-series images capturing the direction of motion are selected. The camera intrinsic and extrinsic parameters of each camera are calculated by combining the camera height and the time-series images. Alternatively, the motion velocity of any one camera is calculated based on its frame rate and the time-series images. Using any complete calibration cloth captured by each camera as the origin of its respective system coordinate system, the offset of each camera is calculated based on the timestamps of all the time-series images, thus obtaining the correct camera extrinsic parameters.

[0139] In one possible implementation, see Figure 5 As shown, camera parameter calibration methods may include:

[0140] Step 501: Set up multiple cameras according to application requirements.

[0141] Step 502: Measure the installation height of each 2D camera.

[0142] Step 503: Place the calibration cloth and control the measuring object to move at a constant speed.

[0143] Step 504: Each camera acquires temporal images, such as RGB or grayscale images, at a high frame rate. For example, each camera can acquire multiple frames (e.g., at least two frames) of complete calibration cloth images.

[0144] Step 505: Filter the complete calibration cloth image for each camera, that is, retain only the complete calibration cloth time sequence image in the motion direction, that is, obtain multiple frames of calibration two-dimensional images.

[0145] Step 506: Calculate the extrinsic parameters and motion velocity of the 3D camera in the four system coordinate systems.

[0146] Step 507: Calculate the intrinsic parameters, extrinsic parameters, and motion velocity of the two-dimensional camera in the four system coordinate systems.

[0147] Step 508: Filter out all cameras with consistent motion speed.

[0148] Step 509: Filter out the correct extrinsic parameters for the 3D camera and the correct intrinsic and extrinsic parameters for the 2D camera.

[0149] For example, based on the constraint of consistent motion speed (i.e., the motion speed of the measured object), the correct (internal) and external parameters are selected from the parameters corresponding to the four system coordinate systems for each camera to complete the calibration.

[0150] Step 510: Perform information association between multiple cameras.

[0151] Based on the same concept as the above method, this application proposes a camera parameter calibration device. A calibration object is placed on the measurement object, and the calibration object includes multiple calibration points. (See [link to relevant documentation]). Figure 6 The diagram shown is a structural schematic of the device. During the movement of the object being measured, the device may include:

[0152] The acquisition module 61 is used to acquire a calibration two-dimensional image of the measurement object using a two-dimensional camera, determine the pixel coordinates of the calibration point in the image coordinate system based on the calibration two-dimensional image, determine the physical coordinates of the calibration point in each system coordinate system based on multiple configured system coordinate systems, and generate multiple coordinate point pairs corresponding to each system coordinate system, the coordinate point pairs including the pixel coordinates of the calibration point and the physical coordinates of the calibration point in the system coordinate system; the determination module 62 is used to determine the camera intrinsic parameters and camera extrinsic parameters of the two-dimensional camera based on the multiple coordinate point pairs corresponding to the system coordinate system, and determine the motion speed of the measurement object based on the camera extrinsic parameters; the calibration module 63 is used to select a target system coordinate system from all system coordinate systems based on the motion speed corresponding to the system coordinate system, and calibrate the camera intrinsic parameters and camera extrinsic parameters corresponding to the target system coordinate system for the two-dimensional camera.

[0153] For example, when determining the camera intrinsic and extrinsic parameters of the two-dimensional camera based on multiple coordinate points corresponding to the system coordinate system, the determining module 62 is specifically used for: configuring an initial focal length value for the camera focal length; determining the target parameter value corresponding to the parameter to be calibrated of the two-dimensional camera based on the initial focal length value and multiple coordinate points corresponding to the system coordinate system; determining the target focal length value corresponding to the camera focal length of the two-dimensional camera based on the target parameter value, the vertical height between the two-dimensional camera and the measurement object, and the initial focal length value; and determining the camera intrinsic and extrinsic parameters of the two-dimensional camera based on the target focal length value corresponding to the camera focal length and the target parameter value corresponding to the parameter to be calibrated.

[0154] For example, when determining the target focal length value corresponding to the camera focal length of the two-dimensional camera based on the target parameter value, the vertical height between the two-dimensional camera and the measurement object, and the initial focal length value, the determining module 62 is specifically used to: determine the candidate focal length value corresponding to the camera focal length of the two-dimensional camera based on the target parameter value, the vertical height between the two-dimensional camera and the measurement object, and the initial focal length value; if the difference between the candidate focal length value and the initial focal length value is less than a preset threshold, then the candidate focal length value is determined as the target focal length value corresponding to the camera focal length; otherwise, the candidate focal length value is determined as the initial focal length value, and the operation of determining the target parameter value corresponding to the calibration parameter of the two-dimensional camera based on the initial focal length value and the multiple coordinate points is returned.

[0155] For example, the parameters to be calibrated include camera center, distortion coefficient, rotation parameters, and translation parameters, wherein the translation parameters include X-axis translation parameters, Y-axis translation parameters, and Z-axis translation parameters; when the determining module 62 determines the candidate focal length value corresponding to the camera focal length of the two-dimensional camera based on the target parameter value, the vertical height between the two-dimensional camera and the measurement object, and the initial focal length value, it is specifically used to: determine an adjustment factor based on the target parameter value corresponding to the vertical height and the Z-axis translation parameter; and adjust the initial focal length value based on the adjustment factor to obtain the candidate focal length value.

[0156] For example, the parameters to be calibrated include the camera center, distortion coefficients, rotation parameters, and translation parameters. When the determining module 62 determines the target parameter values ​​corresponding to the parameters to be calibrated for the two-dimensional camera based on the initial focal length value and the multiple coordinate point pairs, it is specifically used to: configure an initial center value for the camera center and configure an initial coefficient value for the distortion coefficients; determine the initial values ​​of the rotation parameters and translation parameters corresponding to the rotation parameters based on the initial focal length value, the initial center value, the initial coefficient values, and the multiple coordinate point pairs; and optimize the initial center value, the initial coefficient values, the initial values ​​of the rotation parameters, and the initial values ​​of the translation parameters to obtain the target center value corresponding to the camera center, the target coefficient value corresponding to the distortion coefficients, the target value of the rotation parameters corresponding to the rotation parameters, and the target value of the translation parameters corresponding to the translation parameters.

[0157] For example, the calibration two-dimensional image includes a first calibration two-dimensional image and a second calibration two-dimensional image during the movement of the measured object; when the determining module 62 determines the movement speed of the measured object based on the camera extrinsic parameters, it is specifically used to: if the camera extrinsic parameters corresponding to the first calibration two-dimensional image include a first X-axis translation parameter value and a first Y-axis translation parameter value, and the camera extrinsic parameters corresponding to the second calibration two-dimensional image include a second X-axis translation parameter value and a second Y-axis translation parameter value, then determine the distance change based on the first X-axis translation parameter value, the first Y-axis translation parameter value, the second X-axis translation parameter value, and the second Y-axis translation parameter value; determine the movement speed of the measured object based on the distance change and the target duration, wherein the target duration is the time interval between the first calibration two-dimensional image and the second calibration two-dimensional image.

[0158] For example, when the calibration module 63 selects the target system coordinate system from all system coordinate systems based on the motion speed corresponding to each system coordinate system, it is specifically used to: select the motion speed corresponding to all cameras as the target motion speed based on the motion speed corresponding to each system coordinate system of the two-dimensional camera and the motion speed corresponding to each system coordinate system of the other cameras besides the two-dimensional camera, and determine the system coordinate system corresponding to the target motion speed as the target system coordinate system.

[0159] Based on the same concept as the above method, this application proposes an electronic device, see [link to previous application]. Figure 7 As shown, the electronic device may include: a processor 71 and a machine-readable storage medium 72, the machine-readable storage medium 72 storing machine-executable instructions that can be executed by the processor 71; the processor 71 is used to execute the machine-executable instructions to implement the camera parameter calibration method disclosed in the above example of this application.

[0160] Based on the same concept as the above method, this application also provides a machine-readable storage medium storing a plurality of computer instructions, which, when executed by a processor, can implement the camera parameter calibration method disclosed in the above examples of this application.

[0161] The aforementioned machine-readable storage medium can be any electronic, magnetic, optical, or other physical storage device that can contain or store information, such as executable instructions, data, etc. For example, machine-readable storage media can be: RAM (Random Access Memory), volatile memory, non-volatile memory, flash memory, storage drives (such as hard disk drives), solid-state drives, any type of storage disk (such as optical discs, DVDs, etc.), or similar storage media, or combinations thereof.

[0162] The systems, devices, modules, or units described in the above embodiments can be implemented by a computer entity or by a product with a certain function. A typical implementation device is a computer, which can be a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email sending and receiving device, game console, tablet computer, wearable device, or any combination of these devices.

[0163] For ease of description, the above devices are described separately by function as various units. Of course, in implementing this application, the functions of each unit can be implemented in one or more software and / or hardware.

[0164] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, embodiments of this application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0165] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0166] Furthermore, these computer program instructions can also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in the process. Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0167] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0168] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A method for calibrating camera parameters, characterized in that, A calibration object is placed on the object being measured, the calibration object comprising multiple calibration points. During the movement of the object being measured, the method includes: A 2D image of the calibration object is acquired using a 2D camera. The pixel coordinates of the calibration point in the image coordinate system are determined based on the calibration image. The physical coordinates of the calibration point in each system coordinate system are determined based on the configured multiple system coordinate systems. For each system coordinate system, multiple coordinate point pairs corresponding to the system coordinate system are generated. The coordinate point pairs include the pixel coordinates of the calibration point and the physical coordinates of the calibration point in the system coordinate system. The camera intrinsic and extrinsic parameters of the two-dimensional camera are determined based on multiple coordinate point pairs corresponding to the system coordinate system, and the motion velocity of the measured object is determined based on the camera extrinsic parameters. Based on the motion velocity corresponding to each system coordinate system, a target system coordinate system is selected from all system coordinate systems to calibrate the camera intrinsic and extrinsic parameters corresponding to the target system coordinate system for the two-dimensional camera; The step of determining the camera intrinsic and extrinsic parameters of the two-dimensional camera based on multiple coordinate point pairs corresponding to the system coordinate system includes: configuring an initial focal length value for the camera focal length; determining a target parameter value corresponding to the parameter to be calibrated of the two-dimensional camera based on the initial focal length value and the multiple coordinate point pairs; determining a target focal length value corresponding to the camera focal length of the two-dimensional camera based on the target parameter value, the vertical height between the two-dimensional camera and the measurement object, and the initial focal length value; and determining the camera intrinsic and extrinsic parameters of the two-dimensional camera based on the target focal length value corresponding to the camera focal length and the target parameter value corresponding to the parameter to be calibrated.

2. The method according to claim 1, characterized in that, The step of determining the target focal length value corresponding to the camera focal length of the two-dimensional camera based on the target parameter value, the vertical height between the two-dimensional camera and the measurement object, and the initial focal length value includes: Based on the target parameter value, the vertical height between the two-dimensional camera and the measurement object, and the initial focal length value, the candidate focal length value corresponding to the camera focal length of the two-dimensional camera is determined; If the difference between the candidate focal length value and the initial focal length value is less than a preset threshold, then the candidate focal length value is determined as the target focal length value corresponding to the camera focal length; otherwise, the candidate focal length value is determined as the initial focal length value, and the operation of determining the target parameter value corresponding to the calibration parameter of the two-dimensional camera based on the initial focal length value and the multiple coordinate point pairs is returned.

3. The method according to claim 2, characterized in that, The parameters to be calibrated include camera center, distortion coefficient, rotation parameters, and translation parameters. The translation parameters include X-axis translation parameters, Y-axis translation parameters, and Z-axis translation parameters. The step of determining the candidate focal length value corresponding to the camera focal length of the two-dimensional camera based on the target parameter value, the vertical height between the two-dimensional camera and the measurement object, and the initial focal length value includes: The adjustment factor is determined based on the target parameter values ​​corresponding to the vertical height and the Z-axis translation parameter; The initial focal length value is adjusted based on the adjustment factor to obtain the candidate focal length value.

4. The method according to claim 1, characterized in that, The parameters to be calibrated include the camera center, distortion coefficients, rotation parameters, and translation parameters. Determining the target parameter values ​​corresponding to the parameters to be calibrated for the 2D camera based on the initial focal length value and the multiple coordinate point pairs includes: Configure an initial center value for the camera center and an initial coefficient value for the distortion coefficients; Based on the initial focal length value, the initial center value, the initial coefficient value, and the multiple coordinate point pairs, determine the initial values ​​of the rotation parameters corresponding to the rotation parameters and the initial values ​​of the translation parameters corresponding to the translation parameters; The initial center value, the initial coefficient value, the initial value of the rotation parameter, and the initial value of the translation parameter are optimized to obtain the target center value corresponding to the camera center, the target coefficient value corresponding to the distortion coefficient, the target value of the rotation parameter corresponding to the rotation parameter, and the target value of the translation parameter corresponding to the translation parameter.

5. The method according to any one of claims 1-4, characterized in that, The calibration two-dimensional image includes a first calibration two-dimensional image and a second calibration two-dimensional image during the motion of the measured object; Determining the motion speed of the measured object based on the camera extrinsic parameters includes: If the camera extrinsic parameters corresponding to the first calibration two-dimensional image include the first X-axis translation parameter value and the first Y-axis translation parameter value, and the camera extrinsic parameters corresponding to the second calibration two-dimensional image include the second X-axis translation parameter value and the second Y-axis translation parameter value, then the distance change is determined based on the first X-axis translation parameter value, the first Y-axis translation parameter value, the second X-axis translation parameter value, and the second Y-axis translation parameter value. The motion speed of the measured object is determined based on the distance change and the target duration, where the target duration is the time interval between the first calibrated two-dimensional image and the second calibrated two-dimensional image.

6. The method according to any one of claims 1-4, characterized in that, The step of selecting the target system coordinate system from all system coordinate systems based on the motion velocity corresponding to each system coordinate system includes: Based on the motion velocity corresponding to each system coordinate system of the two-dimensional camera and the motion velocity corresponding to each system coordinate system of the other cameras besides the two-dimensional camera, a target motion velocity corresponding to all cameras is selected, and the system coordinate system corresponding to the target motion velocity is determined as the target system coordinate system.

7. A camera parameter calibration device, characterized in that, A calibration object is placed on the object being measured, the calibration object comprising multiple calibration points. During the movement of the object being measured, the device includes: The acquisition module is used to acquire a calibration two-dimensional image of the measurement object using a two-dimensional camera, determine the pixel coordinates of the calibration point in the image coordinate system based on the calibration two-dimensional image, determine the physical coordinates of the calibration point in each system coordinate system based on multiple configured system coordinate systems, and generate multiple coordinate point pairs corresponding to each system coordinate system, wherein the coordinate point pairs include the pixel coordinates of the calibration point and the physical coordinates of the calibration point in the system coordinate system. The determination module is used to determine the camera intrinsic parameters and camera extrinsic parameters of the two-dimensional camera based on multiple coordinate point pairs corresponding to the system coordinate system, and to determine the motion speed of the measured object based on the camera extrinsic parameters; The calibration module is used to select the target system coordinate system from all system coordinate systems based on the motion velocity corresponding to the system coordinate system, and to calibrate the camera intrinsic and extrinsic parameters corresponding to the target system coordinate system for the 2D camera. Specifically, when determining the camera intrinsic and extrinsic parameters of the 2D camera based on multiple coordinate point pairs corresponding to the system coordinate system, the determining module is used to: configure an initial focal length value for the camera focal length; determine the target parameter value corresponding to the parameter to be calibrated of the 2D camera based on the initial focal length value and the multiple coordinate point pairs; determine the target focal length value corresponding to the camera focal length of the 2D camera based on the target parameter value, the vertical height between the 2D camera and the measurement object, and the initial focal length value; and determine the camera intrinsic and extrinsic parameters of the 2D camera based on the target focal length value corresponding to the camera focal length and the target parameter value corresponding to the parameter to be calibrated.

8. The apparatus according to claim 7, Its features are, Specifically, when the determining module determines the target focal length value corresponding to the camera focal length of the 2D camera based on the target parameter value, the vertical height between the 2D camera and the measurement object, and the initial focal length value, it is used to: determine the candidate focal length value corresponding to the camera focal length of the 2D camera based on the target parameter value, the vertical height between the 2D camera and the measurement object, and the initial focal length value; if the difference between the candidate focal length value and the initial focal length value is less than a preset threshold, then the candidate focal length value is determined as the target focal length value corresponding to the camera focal length; otherwise, the candidate focal length value is determined as the initial focal length value, and the operation of determining the target parameter value corresponding to the calibration parameter of the 2D camera based on the initial focal length value and the multiple coordinate point pairs is returned. The parameters to be calibrated include camera center, distortion coefficients, rotation parameters, and translation parameters. The translation parameters include X-axis translation parameters, Y-axis translation parameters, and Z-axis translation parameters. When the determining module determines the candidate focal length value corresponding to the camera focal length of the two-dimensional camera based on the target parameter value, the vertical height between the two-dimensional camera and the measurement object, and the initial focal length value, it specifically performs the following: determining an adjustment factor based on the target parameter value corresponding to the vertical height and the Z-axis translation parameter; and adjusting the initial focal length value based on the adjustment factor to obtain the candidate focal length value. The parameters to be calibrated include the camera center, distortion coefficients, rotation parameters, and translation parameters. When the determining module determines the target parameter values ​​corresponding to the calibrated parameters of the 2D camera based on the initial focal length value and the multiple coordinate point pairs, it specifically performs the following: configuring an initial center value for the camera center and initial coefficient values ​​for the distortion coefficients; determining initial values ​​for the rotation parameters and translation parameters based on the initial focal length value, initial center value, initial coefficient values, and the multiple coordinate point pairs; and optimizing the initial center value, initial coefficient values, initial rotation parameter values, and initial translation parameter values ​​to obtain the target center value corresponding to the camera center, the target coefficient value corresponding to the distortion coefficients, the target rotation parameter value corresponding to the rotation parameters, and the target translation parameter value corresponding to the translation parameters. The calibration two-dimensional image includes a first calibration two-dimensional image and a second calibration two-dimensional image during the movement of the measured object. When the determining module determines the movement speed of the measured object based on the camera extrinsic parameters, it specifically performs the following: if the camera extrinsic parameters corresponding to the first calibration two-dimensional image include a first X-axis translation parameter value and a first Y-axis translation parameter value, and the camera extrinsic parameters corresponding to the second calibration two-dimensional image include a second X-axis translation parameter value and a second Y-axis translation parameter value, then the distance change is determined based on the first X-axis translation parameter value, the first Y-axis translation parameter value, the second X-axis translation parameter value, and the second Y-axis translation parameter value; the movement speed of the measured object is determined based on the distance change and a target duration, where the target duration is the time interval between the first calibration two-dimensional image and the second calibration two-dimensional image. Specifically, when the calibration module selects the target system coordinate system from all system coordinate systems based on the motion speed corresponding to each system coordinate system, it is used to: select the motion speed corresponding to all cameras as the target motion speed based on the motion speed corresponding to each system coordinate system of the two-dimensional camera and the motion speed corresponding to each system coordinate system of the other cameras besides the two-dimensional camera, and determine the system coordinate system corresponding to the target motion speed as the target system coordinate system.

9. An electronic device, characterized in that, include: A processor and a machine-readable storage medium, the machine-readable storage medium storing machine-executable instructions that can be executed by the processor; The processor is configured to execute the machine-executable instructions to implement the method of any one of claims 1-6.

Images