A method and device for calibrating a profilometer camera and an electronic device

Abstract

Embodiments of the present application provide a kind of profilometer camera calibration method, device and electronic equipment, specifically include: the three-dimensional point cloud data obtained by profilometer camera acquisition calibration block is acquired, for each surface of calibration block, the model constraint condition of each surface is determined, and the local point cloud data belonging to each surface of calibration block is determined from three-dimensional point cloud data, the extrinsic parameter that makes the local point cloud data of each surface and the model constraint condition of each surface match is determined as the target extrinsic parameter of profilometer camera, profilometer camera is calibrated.So, by the embodiment provided in the present application, profilometer camera can be calibrated based on the surface model constraint condition of calibration block with the help of simple structure calibration block, can effectively reduce the calibration cost of profilometer camera, effectively improve the measurement accuracy of profilometer camera.

Description

Technical Field

[0001] This invention relates to the field of optical measurement, and in particular to a calibration method, apparatus, and electronic device for a profilometer camera. Background Technology

[0002] In the field of optical measurement, profilometer cameras are used to measure information such as the contour, two-dimensional dimensions, and depth of an object. If the profilometer camera's measurement accuracy is inaccurate, the validity of the acquired data is likely to be low. Therefore, in this field, standard metrological instruments are typically used to calibrate the accuracy of the profilometer camera; this calibration process is called standardization.

[0003] In related technologies, calibration blocks embedded with steel balls or printed with special patterns are often used. The center of the steel ball or the special pattern in the calibration block is used as a feature point to calibrate the accuracy of the profilometer camera. The calibration blocks used in this calibration method often have a relatively complex structure and high manufacturing cost. Furthermore, a professional measurement institution is required to accurately measure the dimensions of each feature point of the calibration block to ensure its dimensional accuracy, resulting in high calibration costs. Summary of the Invention

[0004] The purpose of this invention is to provide a calibration method, apparatus, and electronic device for a profilometer camera, thereby reducing the calibration cost of the profilometer camera. The specific technical solution is as follows:

[0005] In a first aspect, embodiments of the present invention provide a calibration method for a profilometer camera, characterized in that the method includes:

[0006] Acquire 3D point cloud data obtained from the profilometer camera's acquisition calibration block;

[0007] For each surface of the calibration block, determine the model constraints of the surface and determine the local point cloud data belonging to the surface from the three-dimensional point cloud data;

[0008] The extrinsic parameters that match the local point cloud data of each surface with the model constraints of each surface are determined and used as the target extrinsic parameters of the profilometer camera.

[0009] In conjunction with the first aspect, the present invention provides a second possible embodiment, in which the determination of extrinsic parameters that match the local point cloud data of each surface with the model constraints of each surface, as the target extrinsic parameters of the profilometer camera, includes:

[0010] Based on the model constraints of each surface, the correspondence between model constraint error and extrinsic parameters is determined, wherein the model constraint error is: the error of the local point cloud data of each surface relative to the model constraints of each surface when the extrinsic parameters of the profilometer camera are the extrinsic parameters corresponding to the model constraint error;

[0011] Based on the correspondence, the extrinsic parameter corresponding to the minimum value of the model constraint error is determined and used as the target extrinsic parameter of the profilometer camera.

[0012] In conjunction with the first aspect, the present invention provides a third possible embodiment, in which the step of determining local point cloud data belonging to each surface of the calibration block from the three-dimensional point cloud data includes:

[0013] The point cloud data in the three-dimensional point cloud data is fitted to obtain the features of the fitting curve to which each point cloud data belongs, which are used as the surface features of the point cloud data.

[0014] For each surface, according to the preset correspondence between surface features and surfaces, the point cloud data corresponding to the surface features is determined from the three-dimensional point cloud data, and used as the local point cloud data of the surface.

[0015] In conjunction with the third possible embodiment of the first aspect, the present invention provides a fourth possible embodiment, in which the fitting of the point cloud data in the three-dimensional point cloud data includes:

[0016] Based on the preset shape of the calibration block, determine the type of each contour line of the calibration block;

[0017] The point cloud data in the three-dimensional point cloud data is fitted to a fitting curve of the type mentioned above, and the characteristics of the fitting curve to which each point cloud data belongs are obtained, which are used as the surface features of the point cloud data.

[0018] In conjunction with the first aspect, the present invention provides a fifth possible embodiment, in which determining the model constraints of the surface includes:

[0019] The surface equation of the surface is determined based on the preset size and preset shape of the surface, and serves as the model constraint condition for the surface.

[0020] In conjunction with the first aspect, the present invention provides a sixth possible embodiment in which the number of calibration blocks is multiple;

[0021] The determination of the model constraints for the surface includes:

[0022] Based on the preset size, preset shape, and preset arrangement of each calibration block, the surface equation of the surface is determined as the model constraint condition of the surface.

[0023] In conjunction with the first aspect, the present invention provides a seventh possible embodiment, wherein acquiring the 3D point cloud data obtained by the profilometer camera from the calibration block includes:

[0024] Acquire multiple frames of 3D point cloud data obtained by the profilometer camera from calibration blocks located at different positions;

[0025] Determining local point cloud data belonging to the surface from the three-dimensional point cloud data includes:

[0026] Local point cloud data belonging to the surface are determined from each frame of 3D point cloud data.

[0027] Secondly, embodiments of the present invention provide a calibration device for a profilometer camera, characterized in that the device comprises:

[0028] The acquisition module is used to acquire 3D point cloud data obtained by the profilometer camera from the calibration block.

[0029] The first determining module is used to determine the model constraints of each surface of the calibration block; and to determine the local point cloud data belonging to the surface from the three-dimensional point cloud data.

[0030] The second determining module is used to determine the extrinsic parameters that match the local point cloud data of each surface with the model constraints of each surface, and use these as the target extrinsic parameters of the profilometer camera.

[0031] In conjunction with the second aspect, the present invention provides a second possible embodiment, in which the second determining module is specifically used for:

[0032] Based on the model constraints of each surface, the correspondence between model constraint error and extrinsic parameters is determined, wherein the model constraint error is: the error of the local point cloud data of each surface relative to the model constraints of each surface when the extrinsic parameters of the profilometer camera are the extrinsic parameters corresponding to the model constraint error;

[0033] Based on the correspondence, the extrinsic parameter corresponding to the minimum value of the model constraint error is determined and used as the target extrinsic parameter of the profilometer camera.

[0034] In conjunction with the second aspect, the present invention provides a third possible embodiment, in which the first determining module is specifically used for:

[0035] The point cloud data in the three-dimensional point cloud data is fitted to obtain the features of the fitting curve to which each point cloud data belongs, which are used as the surface features of the point cloud data.

[0036] For each surface, according to the preset correspondence between surface features and surfaces, the point cloud data corresponding to the surface features is determined from the three-dimensional point cloud data, and used as the local point cloud data of the surface.

[0037] In conjunction with the third possible embodiment of the second aspect, the present invention provides a fourth possible embodiment, in which the fitting of the point cloud data in the three-dimensional point cloud data includes:

[0038] Based on the preset shape of the calibration block, determine the type of each contour line of the calibration block;

[0039] The point cloud data in the three-dimensional point cloud data is fitted to a fitting curve of the type mentioned above, and the characteristics of the fitting curve to which each point cloud data belongs are obtained, which are used as the surface features of the point cloud data.

[0040] In conjunction with the second aspect, the present invention provides a fifth possible embodiment, in which the first determining module is further configured to:

[0041] The surface equation of the surface is determined based on the preset size and preset shape of the surface, and serves as the model constraint condition for the surface.

[0042] In conjunction with the second aspect, the present invention provides a sixth possible embodiment, in which the number of calibration blocks is multiple; the first determining module is further configured to:

[0043] The surface equation of the surface is determined based on the preset size, preset shape, and preset arrangement of each calibration block, and serves as the model constraint condition for the surface.

[0044] In conjunction with the second aspect, the present invention provides a seventh possible embodiment, wherein the acquisition module is used to acquire multiple frames of three-dimensional point cloud data obtained by the profilometer camera from calibration blocks located at different positions;

[0045] The first determining module is used to determine local point cloud data belonging to the surface from each frame of three-dimensional point cloud data.

[0046] Thirdly, embodiments of the present invention provide an electronic device, the electronic device including a processor and a memory, wherein the memory is used to store computer programs; the processor is used to execute the program stored in the memory to implement the calibration method steps of the profilometer camera described in the first aspect.

[0047] Fourthly, this embodiment provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the calibration method steps of the profilometer camera described in the first aspect.

[0048] Beneficial effects of the embodiments of the present invention:

[0049] This invention provides a calibration method, apparatus, and device for a profilometer camera. Specifically, the method includes acquiring 3D point cloud data obtained by the profilometer camera from a calibration block; determining model constraints for each surface of the calibration block; identifying local point cloud data belonging to each surface of the calibration block from the 3D point cloud data; and determining extrinsic parameters that match the local point cloud data of each surface with the model constraints of each surface, which are then used as the target extrinsic parameters for the profilometer camera. Since the model constraints of each surface of the calibration block are determined based on the actual surface dimensions of the calibration block, the simpler the shape of the calibration block, the simpler and more accurate the process for determining the model constraints of each surface.

[0050] Thus, the profilometer camera calibration method provided in this embodiment of the invention can directly adjust the extrinsic parameters of the profilometer camera based on the model constraints of the calibration block's surface, using a simple calibration block structure. This ensures that the 3D point cloud data acquired by the adjusted profilometer camera matches the actual surface constraints of the calibration block, thereby calibrating the profilometer camera. This effectively reduces the calibration cost of the profilometer camera while improving its measurement accuracy.

[0051] Of course, implementing any product or method of the present invention does not necessarily require achieving all of the advantages described above at the same time. Attached Figure Description

[0052] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other embodiments can be obtained based on these drawings.

[0053] Figure 1 A possible flowchart illustrating the profilometer camera calibration method provided by the present invention;

[0054] Figure 2a A schematic diagram of a possible process for determining local point cloud data provided by the present invention;

[0055] Figure 2b A schematic diagram of a possible fitting curve feature provided by the present invention;

[0056] Figure 3a A possible flowchart for determining the surface features of point cloud data provided by the present invention;

[0057] Figure 3b A schematic diagram illustrating another possible fitting curve feature provided by the present invention;

[0058] Figure 4 This is a schematic diagram of a possible calibration block shape provided by the present invention;

[0059] Figure 5 A schematic diagram of a possible surface model provided by the present invention;

[0060] Figure 6 This is a schematic diagram of a possible calibration block arrangement provided by the present invention;

[0061] Figure 7 This invention provides a schematic diagram of a possible method for mounting multiple profilometer cameras.

[0062] Figure 8 This is a schematic diagram of another possible arrangement of calibration blocks provided by the present invention;

[0063] Figure 9 A schematic diagram illustrating another possible mounting method for the multi-profilometer camera provided by the present invention;

[0064] Figure 10 A schematic diagram of a possible structure of the profilometer camera calibration device provided by the present invention;

[0065] Figure 11 This is a schematic diagram of a possible structure of the electronic device provided by the present invention. Detailed Implementation

[0066] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art based on the present invention are within the scope of protection of the present invention.

[0067] In related technologies, the principle of calibrating a profilometer camera using a calibration block embedded with steel balls or printed with special patterns is as follows: using the steel balls or special patterns on the calibration block as feature points, the profilometer camera scans the calibration block to obtain 3D point cloud data of the calibration block. Registration is performed using the positional relationship between the coordinates of the feature points extracted from the 3D point cloud data and the actual feature points of the calibration block. The extrinsic parameters of the profilometer camera are adjusted using the registration results so that the 3D point cloud data acquired by the adjusted profilometer camera can accurately represent the actual size of the calibration block.

[0068] Using calibration blocks inlaid with steel balls or printed with special patterns has two main drawbacks. First, the complex structure of these blocks inevitably leads to long processing cycles and high costs. Second, because the steel balls or special patterns serve as feature points, the dimensions of these feature points, such as the coordinates of the steel ball's center and its diameter, must be precisely determined before calibrating the profilometer camera. This means that the processed calibration blocks must be sent to a professional precision measurement institution for dimensional measurement, and a model coordinate system must be constructed based on the designed calibration blocks to obtain accurate measurement reference data. This inevitably increases the measurement cycle and costs, thus raising the overall calibration cost of the profilometer camera.

[0069] In view of this, in order to save the calibration cost of profilometer cameras, the present invention provides a profilometer camera calibration method, which can be applied to any electronic device that supports profilometer camera calibration, including but not limited to mobile terminals, personal computers or servers, and can also be any measurement system that supports profilometer camera calibration. This embodiment does not impose any limitations on this.

[0070] In one possible embodiment, the profilometer camera calibration method provided by the present invention can be as follows: Figure 1 As shown, the specific steps include the following:

[0071] S110. Acquire the 3D point cloud data obtained by the profilometer camera from the calibration block;

[0072] S120. For each surface of the calibration block, determine the surface model constraints and the local point cloud data belonging to that surface;

[0073] S130. Determine the extrinsic parameters that match the local point cloud data of each surface with the model constraints of the surface, and use them as the target extrinsic parameters of the profilometer camera.

[0074] In this embodiment of the invention, the surface model constraints of each actual surface of the calibration block are matched with the local point cloud data of each surface. This is equivalent to using the surface model constraints of the actual surface of the calibration block to constrain the parameters of the profilometer camera, thereby calibrating the profilometer camera. This ensures that the local point cloud data of each surface acquired by the calibrated profilometer camera conforms to the surface model constraints of the actual surface of the calibration block. In other words, the acquired local point cloud data of each surface can accurately represent the actual surface features of the calibration block, thus completing the calibration of the profilometer camera.

[0075] By employing the embodiments of the present invention, the calibration block is not required to have a specific structure or pattern. Therefore, a simple calibration block can be used for profilometer camera calibration, thereby effectively saving the processing cost and measurement cost of the calibration block, and further saving the calibration cost of the profilometer camera.

[0076] To clearly illustrate the profilometer camera calibration method provided in the embodiments of the present invention, the aforementioned steps S110-S130 will be described below:

[0077] In this embodiment of the invention, the profilometer camera can be any device with three-dimensional information acquisition capabilities, including but not limited to line laser profilometers, line laser sensors, 3D scanners, depth cameras, etc. In this embodiment of the invention, the number of profilometer cameras can be one or more; the specific type, model, and number of profilometer cameras can be selected according to actual measurement needs, and this invention does not impose specific limitations.

[0078] In one possible scenario, a profilometer camera performs an optical scan of the object by emitting light of a specific wavelength onto its surface, and then generates 3D point cloud data of the object based on the scan results. The generated 3D point cloud data is essentially a collection of points located in the same spatial reference frame, representing the spatial distribution of the object and the spectral characteristics of its surface.

[0079] Specifically, in step S110, data can be acquired from the calibration block using a profilometer camera to obtain its 3D point cloud data. This 3D point cloud data can be understood as follows: in the image coordinate system of the profilometer camera, there exists a set consisting of a massive number of points. The distribution of these points characterizes the spatial distribution and surface spectral properties of the calibration block. The spatial distribution of the calibration block includes its shape information, which can specifically include its size and surface distribution. The surface distribution can include the number of surfaces, the characteristics of the surfaces, and the spatial relationships between the surfaces. The surface spectral properties of the calibration block reflect its surface roughness and material composition.

[0080] Since the 3D point cloud data of the calibration block obtained in step S110 is a collection of massive points, the distribution of each point can characterize the spatial distribution and surface spectral properties of the calibration block. In step S120, for each surface of the calibration block, the local point cloud data belonging to each surface of the calibration block can be determined. Specifically, according to the spatial distribution of the calibration block, the points in the 3D point cloud data of the calibration block obtained in step S110 are divided, and points belonging to the same surface are divided into a point cloud data set, thus obtaining the local point cloud data of each surface.

[0081] For example, taking a cone as the calibration block, data is collected from the calibration block using a profilometer camera, resulting in a three-dimensional point cloud composed of a large number of points. This three-dimensional point cloud can characterize the spatial distribution of the calibration block; that is, the three-dimensional point cloud can represent that the calibration block is a spatial structure composed of a cone surface and a circle at the base. Thus, based on the spatial distribution of the calibration block, the surface affiliation of each point cloud can be determined, and points belonging to the same surface can be grouped into a point cloud set. Specifically, using step S120, points belonging to the cone surface in the three-dimensional point cloud can be identified as the first local point cloud data, and the remaining points belonging to the circle at the base can be identified as the second local point cloud data.

[0082] In one possible embodiment, such as Figure 2a As shown, in step S120, for each surface of the calibration block, determining the local point cloud data belonging to that surface from the 3D point cloud data can be achieved through the following steps:

[0083] S121, Fit the point cloud data in the three-dimensional point cloud data to obtain the features of the fitting curve to which each point cloud data belongs, and use it as the surface features of the point cloud data.

[0084] The curves used in this article do not specifically refer to bent lines, but also include straight lines. Similarly, the surfaces used in this article can be of any shape; that is, the surface can be a plane or a curved surface, and this invention does not impose any limitations on this.

[0085] It is understandable that different surfaces have different shapes, and therefore curves on different surfaces will have different shapes. For example, taking a calibration block as a cone, the base of the cone is a circular plane, so the line connecting any two points on the base is a straight line. However, the curves on the sides of the cone are conic sections, so the lines connecting two points on the sides are conic sections. In other words, if point cloud data belonging to the base is fitted, the resulting fitted curve will be a straight line, while if point cloud data belonging to the sides is fitted, the resulting fitted curve will be a conic section. For example,... Figure 2b As shown. Since the fitted curves obtained from fitting point cloud data of different surfaces have different shapes, these shapes can be regarded as features of these fitted curves. At the same time, as analyzed above, these features depend on the surface to which the point cloud data belongs. Therefore, these features can reflect the surface of the point cloud data to a certain extent. Therefore, these features are referred to as surface features in this paper.

[0086] S122, for each surface, according to the preset correspondence between surface features and surfaces, determine the surface features and the corresponding point cloud data from the three-dimensional point cloud data, and use them as the local point cloud data of the surface.

[0087] As analyzed above, the surface features of point cloud data depend on the surface to which the point cloud data belongs. Therefore, point cloud data for a specific surface will possess specific surface features, meaning there is a correspondence between surfaces and surface features. Based on this correspondence and the surface features obtained through fitting, the local point cloud data for each surface can be determined. For example, still using the cone example mentioned earlier, if the surface features of the point cloud data indicate that the fitted curve obtained by fitting the point cloud data is a conic section, then the point cloud data belongs to the lateral surface of the cone. Conversely, if the surface features of the point cloud data indicate that the fitted curve obtained by fitting the point cloud data is a straight line, then the point cloud data belongs to the base of the conic section.

[0088] By using this embodiment, point cloud data can be fitted to obtain the surface features of each point cloud data. By utilizing the characteristic that surface features depend on the surface, the surface to which each point cloud data belongs can be accurately determined based on the surface features of the point cloud data, that is, the local point cloud data of each surface can be accurately determined.

[0089] In the aforementioned S121, different methods can be used for fitting depending on the application scenario. For example, in one possible embodiment, the three-dimensional point cloud data is fitted into curves of various different shapes, and the degree of matching between the three-dimensional point cloud data and each fitted curve is calculated. The curve with the highest degree of matching is taken as the fitting curve of the three-dimensional point cloud data.

[0090] In another possible embodiment, the above S121, as Figure 3a As shown, it can also be achieved through S1211-S1212:

[0091] S1211, Determine the type of the outline of the calibration block according to the preset shape of the calibration block.

[0092] The outline of the calibration block is the projection of the calibration block's surface onto a cross-section. This cross-section is any plane perpendicular to the platform used to place the calibration block. It is understood that the type of outline of the same calibration block will differ depending on the cross-section. For example, ... Figure 3b As shown, Figure 3b The upper left section shows the contour lines of different cross-sections when the calibration block is in the shape of a trapezoidal frustum. Figure 3b The upper right section shows the contour lines of different sections when the calibration block is shaped like a crater. Figure 3b The lower left section shows the contour lines of different cross-sections when the calibration block is cone-shaped. Figure 3b The lower right section shows the contour lines of different sections when the calibration block is spherical.

[0093] S1212, fits the point cloud data in the three-dimensional point cloud data into fitting curves of various contour types, and obtains the features of the fitting curve to which each point cloud data belongs, which are used as the surface features of the point cloud data.

[0094] The 3D point cloud data acquired by the profilometer camera is the point cloud data of the calibration block surface. Therefore, theoretically, the fitting curve obtained by fitting the 3D point cloud data should be the contour line of the calibration block, that is, the type of the fitting curve should be the same as the type of the contour line of the calibration block. Therefore, determining the type of the obtained contour line can be regarded as the type of the fitting curve.

[0095] As can be seen, by using this embodiment, the type of the fitting curve can be determined before fitting the point cloud data based on the prior knowledge of the preset shape of the calibration block, thereby improving the fitting efficiency of the point cloud data. For example, compared to the aforementioned method of fitting 3D point cloud data into multiple different shapes of curves and then determining the fitting curve from the multiple fitted curves, this embodiment does not require fitting the same point cloud data into multiple different shapes of curves, thus achieving higher fitting efficiency.

[0096] In step S130, model constraints are used to characterize the conditions that points located on the same surface in a spatial coordinate system should satisfy. For example, since points on a surface are located on that surface, each point on the surface should satisfy the surface equation of that surface. The surface equation is an equation used to describe the shape of the surface, so the surface equation of the surface can be used as a model constraint. As another example, if surface 1 is parallel to surface 2, and the distance between surface 1 and surface 2 is s, then the distance from each point on surface 1 to surface 2 should also be s. Therefore, the distance to surface 2, s, can also be used as a model constraint.

[0097] Understandably, since the surface equation depends on the surface's size and shape, and the distance between surfaces depends on the pose of each calibration block, and when the calibration blocks are fixed, the surface size and shape are fixed, but the pose is limited by the specific placement of the calibration blocks and is difficult to know in advance. Therefore, compared to using the distance to the surface as a model constraint, using the surface equation determined based on the surface's size and shape as a model constraint is easier to implement. That is, using the surface equation as a model constraint can effectively reduce the complexity of the profilometer camera calibration method.

[0098] Taking a cone block as the calibration block as an example, the surface equation of the bottom surface of the cone block model is calculated based on the actual bottom diameter of the cone block, which serves as the model constraint condition for the bottom surface. The surface equation of the side surface of the cone block model is determined based on the actual bottom diameter and height of the cone block, which serves as the model constraint condition for the side surface.

[0099] It is understandable that the model constraints depend on the calibration block; therefore, the model constraints are conditions in a coordinate system stationary relative to the calibration block (hereinafter referred to as the calibration block coordinate system). For example, taking the surface equation as a model constraint, this surface equation is an equation in a coordinate system stationary relative to the calibration block. The local point cloud data is point cloud data acquired by the profilometer camera, and therefore is point cloud data in a coordinate system stationary relative to the profilometer camera (hereinafter referred to as the sensor coordinate system). The transformation relationship between the sensor coordinate system and the calibration block coordinate system depends on the extrinsic parameters of the profilometer camera. Therefore, the coordinates of the local point cloud data in the calibration block coordinate system depend on the extrinsic parameters. Since the model constraints are conditions in the calibration block coordinate system, the coordinates of the local point cloud data in the calibration block coordinate system directly affect whether the local point cloud data matches the model constraints.

[0100] For cases where multiple calibration blocks exist, for example, such as Figure 4 As shown, since the positions of different calibration blocks are different, in one possible embodiment, different calibration block coordinate systems can be set for different calibration blocks. For example, for each calibration block, a calibration block coordinate system is constructed with the center of that calibration block as the origin. In this example, the model constraints of the surfaces of different calibration blocks are conditions under different calibration block coordinate systems, therefore the individual model constraints are independent of each other.

[0101] In another possible embodiment, the surface equations of the surfaces are determined based on the preset size, preset shape, and preset arrangement of the calibration blocks, serving as model constraints for the surfaces. This integrates the model constraints of each surface into the same calibration block coordinate system. For example, such as... Figure 5 As shown in the example, the calibration block coordinate system is constructed with the center of the calibration block in the upper left corner as the origin, and the model constraints of each surface of the four calibration blocks are integrated into the calibration block coordinate system.

[0102] It is understandable that constraints also exist between the surfaces. For example, let's continue with... Figure 5 For example, Figure 5 The local point cloud data on the left side of the upper right calibration block is horizontally (i.e., the distance between the left side of the upper left calibration block and the horizontal distance between the left side of the upper left calibration block) Figure 5 The distance in the left and right directions should be dx. Similarly, Figure 5 The local point cloud data on the left side of the lower left calibration block is vertically (i.e., the distance from the left side of the upper left calibration block) to the left side of the calibration block. Figure 5 The distance (vertical direction) should be dy. Yes, by integrating the model constraints of each surface of multiple calibration blocks into the same calibration block coordinate system, the model constraints of one surface will not only constrain the local point cloud data of that surface, but also constrain the local point cloud data of other surfaces. This enriches the model constraints.

[0103] It is understandable that the more constraints a model has, the higher the probability that the target extrinsic parameters that match the local point cloud data with the model constraints are the true extrinsic parameters of the profilometer camera. In other words, by selecting this embodiment, the model constraints are enriched as much as possible by integrating the surface equations into the same calibration block coordinate system, thereby improving the accuracy of the determined target extrinsic parameters, which further improves the accuracy of the profilometer camera calibration.

[0104] Understandable Figure 4 , Figure 5 The arrangement shown is only one possible arrangement. In other possible embodiments, the arrangement of multiple calibration blocks may also be different. Figure 4 , Figure 5 As shown, exemplaryly, in some possible embodiments, the arrangement of multiple calibration blocks can also be as follows: Figure 6 , Figure 7 , Figure 8 as well as Figure 9 As shown in any of the attached figures.

[0105] Furthermore, since a greater number of calibration blocks results in a greater number of surfaces, there will be more model constraints. Therefore, in one possible embodiment, to enrich the model constraints and improve calibration accuracy, a larger number of calibration blocks can be set.

[0106] In another possible embodiment, the calibration block can also be made mobile, meaning that the calibration block is in multiple different positions at multiple different times, for example, such as... Figures 6-9 As shown, the calibration block is placed on a conveyor belt so that it moves under the belt's influence. In this example, multiple frames of 3D point cloud data are obtained by acquiring the calibration block at different positions using a profilometer camera. Local point cloud data belonging to the surface are determined from each frame of 3D point cloud data, and the target pose is determined based on the local point cloud data and the surface constraints of each surface.

[0107] It is understandable that when the calibration block is in different positions, the surface on the calibration block will also be in different positions, and the model constraints of the same surface will also be different in different positions. Therefore, by choosing this embodiment, the model constraints can be enriched by placing the calibration blocks in different positions, thereby improving the accuracy of the profilometer camera calibration. At the same time, it is not necessary to set a large number of calibration blocks, thus effectively reducing the cost of profilometer camera calibration.

[0108] The local point cloud data is the point cloud data of the calibration block surface. Therefore, if the extrinsic parameter is the true extrinsic parameter of the profilometer camera, then the local point cloud data should match the model constraints. Conversely, the extrinsic parameter that makes the local point cloud data match the model constraints can be regarded as the true extrinsic parameter of the profilometer camera. Therefore, in S130, the extrinsic parameter that makes the local point cloud data match the model constraints can be used as the target extrinsic parameter.

[0109] It is understandable that when the local point cloud data of each surface matches the model constraints of the surface, the error between the local point cloud data and the model constraints of the surface is small. Conversely, when the local point cloud data of each surface does not match the model constraints of the surface, the error between the local point cloud data and the model constraints of the surface is large. Therefore, in one possible embodiment, the match between the local point cloud data of each surface and the model constraints of the surface can be determined based on the error between them. For ease of description, the error between the local point cloud data of each surface and the model constraints of the surface will be referred to as the model constraint error.

[0110] For example, in one possible embodiment, the aforementioned S130 can be implemented by S131-S132:

[0111] S131, Based on the model constraint conditions of each surface, determine the correspondence between the model constraint error and the external parameters.

[0112] Wherein, the model constraint error is: the extrinsic parameters of the profilometer camera are the extrinsic parameters corresponding to the model constraint error.

[0113] As analyzed above, the coordinates of local point cloud data in the calibration block coordinate system directly affect whether the local point cloud data matches the model constraints. Therefore, the model constraint error is different under different extrinsic parameters, that is, there is a corresponding relationship between the model constraint error and the extrinsic parameters.

[0114] By using this embodiment, the target extrinsic parameter can be determined based on the established correspondence between the model constraint error and the extrinsic parameter. This allows the determined target extrinsic parameter to be as close as possible to the actual extrinsic parameter of the profilometer camera, thereby further improving the accuracy of the profilometer camera calibration.

[0115] To explain S131 more clearly, the following section will illustrate how to determine the correspondence between model constraint errors and extrinsic parameters using specific application scenarios:

[0116] Assume there are n calibration blocks, each consisting of m surfaces. These n calibration blocks are in motion, such as being placed on a conveyor belt and moving under its influence. A profilometer camera acquires 3D point cloud data from each calibration block at p different times.

[0117] Suppose that the local point cloud data of the k-th surface of the j-th calibration block at time j has coordinates X in the sensor coordinate system. s,i,j,k Then the coordinates of the local point cloud data in the calibration block coordinate system satisfy formula (1):

[0118] X o,i,j,k =f(X) s,i,j,k (1)

[0119] Among them, X o,i,j,k Let f(·) be the coordinates of the local point cloud data in the calibration block coordinate system, and f(·) be the coordinate transformation function between the sensor coordinate system and the calibration block coordinate system. This function depends on the extrinsic parameters of the profilometer camera.

[0120] Let's further assume that the model constraint of the surface described in the local point cloud data is the surface equation of the surface, and that the surface equation of the surface is expressed in the form of formula (2):

[0121] g(X)=0…(2)

[0122] Where g(·) varies depending on the shape of the surface, and X is the coordinate in the calibration block coordinate system. For example, if the surface is a plane, then formula (2) can be rewritten as formula (3):

[0123]

[0124] Where a, b, c, and d are parameters in the surface equation of the plane, and x, y, and z are the x-component, y-component, and z-component of X, respectively.

[0125] If the surface is a cone, then formula (2) can be rewritten as formula (4):

[0126]

[0127] Where D and h are parameters in the surface equation of the conical surface.

[0128] If the surface is spherical, then formula (2) can be rewritten as formula (5):

[0129]

[0130] Where x0, y0, and z0 are the parameters of the surface equation of the sphere.

[0131] X o,i,j,kSubstituting X into the function g(·), we obtain formula (6):

[0132] g(X o,i,j,k ) = e i,j,k …(6)

[0133] Among them, e i,j,k To make X o,i,j,k The result is obtained by inputting into the function g(·). As analyzed above, when X o,i,j,k When matching the model constraints of this surface, e i,j,k It is 0, otherwise, when X o,i,j,k When the model constraints do not match those of the surface, e i,j,k Not equal to 0, and X o,i,j,k The more mismatched e is with the model constraints of this surface i,j,k The larger the value of e, the better. i,j,k It can determine whether the local point cloud data of the surface matches the model constraints of the surface.

[0134] Extending formula (6) to the local point cloud data of each surface, we can obtain formula (7):

[0135]

[0136] Where T represents transpose. As mentioned above regarding e i,j,k Analysis shows that E can represent whether the local point cloud data of each surface matches the model constraints of each surface; in other words, E can be used as the model constraint error. Furthermore, since e i,j,k To make X o,i,j,k The result is obtained by inputting into the function g(·), and X o,i,j,k It is obtained from the function f(·), which depends on the external parameters. Therefore, formula (7) can be regarded as the correspondence between the model constraint error and the external parameters.

[0137] S132, Based on the correspondence, determine the extrinsic parameters corresponding to the minimum model constraint error, and use them as the target extrinsic parameters of the profilometer camera.

[0138] Taking the correspondence as an example where the aforementioned formula (7) is used, the extrinsic parameter that minimizes E is determined as the target extrinsic parameter of the profilometer camera. That is, the target extrinsic parameter should satisfy formula (8):

[0139]

[0140] Among them, T g argmin is the target extrinsic parameter. T This refers to the extrinsic parameter T that minimizes its value, for example, argmin. T E refers to the external parameter T that minimizes the value of E.

[0141] It is understandable that in some application scenarios, the coordinates of local point cloud data in the calibration block coordinate system may not be calculated according to formula (1). For example, in one possible embodiment, they are calculated according to formula (9):

[0142] X o,i,j,k =f OM (X M,i,j,k )…(9)

[0143] Among them, X M,i,j,k Let f be the coordinates of the local point cloud data in the world coordinate system. OM (·) is the coordinate transformation function between the world coordinate system and the calibration block coordinate system. X M,i,j,k The result is obtained by calculation using formula (10):

[0144] X M,i,j,k =f MS (X s,i,j,k )…(10)

[0145] Wherein, function f MS (·) represents the coordinate transformation function between the sensor coordinate system and the world coordinate system. In this embodiment, E depends on the function f. OM (·) and function f MS (·), and assuming that in this embodiment, the step size of the movement of each calibration block between every two moments is step, since the surface equation of each surface depends on step, E also depends on step. In this embodiment, the right-hand side of the first equal sign in formula (8) can be rewritten as formula (11):

[0146]

[0147] Because of function f OM (·) and the step size are fixed values, while the function f MS (·) depends on the external parameters, so formula (11) is still about determining the external parameters that can make the value of E reach the minimum. Therefore, the target external parameters can also be calculated according to formula (11).

[0148] In some application scenarios, there may be multiple profilometer cameras. In one possible embodiment, each profilometer camera can be calibrated according to formula (11). As analyzed above, the more constraints the model has, the more accurate the profilometer camera calibration will be. Therefore, in another possible embodiment, the local point cloud data collected by all profilometer cameras can be integrated into the same calibration block coordinate system, at which point formula (12) can be obtained:

[0149]

[0150] Among them, E r The model constraint error is determined based on the 3D point cloud data acquired by the r-th profilometer camera. For details on how to determine the model constraint error based on 3D point cloud data, please refer to the aforementioned explanations, which will not be repeated here. q represents the number of profilometer cameras, and E... tot This is the sum of the constraint errors of each model determined based on the 3D point cloud data acquired by each profilometer camera. r,i,j,k To make X o,r,i,j,k The result obtained by inputting into the function g(·) is described in the relevant explanation of S131 above. o,r,i,j,k Let be the coordinates of the local point cloud data of the kth surface of the jth calibration block acquired by the rth profilometer camera at time i in the calibration block coordinate system.

[0151] As analyzed above, E r Depends on function f OM (·) and step size step, and depend on function f MSr (·), where the function f MSr (·) is the coordinate transformation function between the sensor coordinate system and the world coordinate system of the r-th profilometer camera. Therefore, E tot Depends on function f OM (·), function f MS1 (·), function f MS2 (·), ..., function f MSq (·) and step size. Wherein, the function f MS1 (·), function f MS2 (·), ..., function f MSq (·) depends on the extrinsic parameters of each profilometer camera, while the function f OM (·) and step size are fixed values. As analyzed above, the model constraint error is minimized when the extrinsic parameters are the true extrinsic parameters of the profilometer camera. Therefore, the extrinsic parameters that minimize the sum of model constraint errors can be considered as the true extrinsic parameters of the profilometer camera. Thus, E can be determined according to formula (13). tot The smallest function f MS1 (·), function f MS2 (·), ..., function f MSq (·):

[0152]

[0153] Then, based on the determined function f MS1 (·), function f MS2 (·), ..., function f MSq(·) Determine the extrinsic parameters of each profilometer camera as the target extrinsic parameters of each profilometer camera. Since formula (11) can only calibrate one profilometer camera at a time, calibrating a profilometer camera according to formula (11) is called single-camera calibration, while formula (13) can calibrate multiple profilometer cameras at once, so calibrating a profilometer camera according to formula (13) is called multi-camera calibration.

[0154] Furthermore, for scenarios with multiple profilometer cameras, not only can each profilometer camera be calibrated separately according to single-camera calibration, or multiple profilometer cameras can be calibrated simultaneously according to multi-camera calibration, but also each profilometer camera can be calibrated separately according to single-camera calibration first, and the extrinsic parameters of each profilometer camera obtained from the calibration can be used as the initial values ​​of the extrinsic parameters of each profilometer camera in formula (13) to determine E tot The smallest function f MS1 (·), function f MS2 (·), ..., function f MSq (·), and based on the determined function f MS1 (·), function f MS2 (·), ..., function f MSq (·) Determine the extrinsic parameters of each profilometer camera, and use them as the target extrinsic parameters of each profilometer camera.

[0155] On the other hand, such as Figure 10 As shown, this embodiment of the invention also provides a calibration device for a profilometer camera, wherein the calibration device includes:

[0156] The acquisition module 101 is used to acquire the three-dimensional point cloud data obtained by the profilometer camera from the calibration block;

[0157] The first determining module 102 is used to determine the model constraints of each surface for each surface of the calibration block, and to determine the local point cloud data belonging to each surface from the three-dimensional point cloud data.

[0158] The second determining module 103 is used to determine the extrinsic parameters that match the local point cloud data of each surface with the model constraints of each surface, and use them as the target extrinsic parameters of the profilometer camera.

[0159] In one possible embodiment, the second determining module 103 is specifically used for:

[0160] Based on the model constraints of each surface, the correspondence between model constraint error and extrinsic parameters is determined, wherein the model constraint error is: the error of the local point cloud data of each surface relative to the model constraints of each surface when the extrinsic parameters of the profilometer camera are the extrinsic parameters corresponding to the model constraint error;

[0161] Based on the correspondence, the extrinsic parameter corresponding to the minimum value of the model constraint error is determined and used as the target extrinsic parameter of the profilometer camera.

[0162] In one possible embodiment, the first determining module 102 is specifically used for:

[0163] The point cloud data in the three-dimensional point cloud data is fitted to obtain the features of the fitted curve of each point cloud data, which are used as the surface features of the point cloud data.

[0164] For each surface, according to the preset correspondence between surface features and surfaces, the point cloud data corresponding to the surface features are determined from the 3D point cloud data, which are used as the local point cloud data of the surface.

[0165] In one possible embodiment, fitting point cloud data in 3D point cloud data includes:

[0166] Based on the preset shape of the calibration block, determine the type of each contour line of the calibration block;

[0167] The point cloud data in the 3D point cloud data is fitted to a type of fitting curve, and the characteristics of the fitting curve to which each point cloud data belongs are obtained, which are used as the surface features of the point cloud data.

[0168] In one possible embodiment, the first determining module 102 is further configured to:

[0169] The surface equation of the surface is determined based on the preset size and preset shape of the surface, and serves as the model constraint condition for the surface.

[0170] In one possible embodiment, the number of calibration blocks is multiple, and the first determining module 102 is further configured to:

[0171] Based on the preset size, preset shape, and preset arrangement of each calibration block, the surface equation of the surface is determined as the model constraint condition of the surface.

[0172] In one possible embodiment, the acquisition module 101 is used to acquire multiple frames of three-dimensional point cloud data obtained by the profilometer camera from calibration blocks located at different positions; the first determination module 102 is further used to determine the local point cloud data belonging to each surface of the calibration block from each frame of three-dimensional point cloud data.

[0173] This invention also provides an electronic device, such as... Figure 11 As shown, it includes a memory 111 for storing computer programs;

[0174] When processor 112 executes a program stored in memory 111, it performs the following steps:

[0175] Acquire 3D point cloud data from the calibration block captured by the profilometer camera;

[0176] For each surface of the calibration block, determine the model constraints of the surface and identify the local point cloud data belonging to the surface from the 3D point cloud data;

[0177] The extrinsic parameters that match the local point cloud data of each surface with the model constraints of each surface are determined and used as the target extrinsic parameters of the profilometer camera.

[0178] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0179] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0180] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0181] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0182] In another embodiment of the present invention, a computer-readable storage medium is also provided, which stores a computer program that, when executed by a processor, implements the steps of any of the above-described profilometer camera calibration methods.

[0183] In another embodiment of the present invention, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute any of the profilometer camera calibration methods described in the above embodiments.

[0184] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk (SSD)).

[0185] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0186] The various embodiments in this specification are described in a related manner. For the same or similar parts between the various embodiments, please refer to each other. Each embodiment focuses on describing the differences from other embodiments.

[0187] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.

Claims

1. A calibration method for a profilometer camera, characterized in that, The method includes: Acquire 3D point cloud data obtained from the profilometer camera's acquisition calibration block; For each surface of the calibration block, determine the model constraints of the surface and determine the local point cloud data belonging to the surface from the three-dimensional point cloud data; The extrinsic parameters that match the local point cloud data of each surface with the model constraints of each surface are determined and used as the target extrinsic parameters of the profilometer camera. The number of calibration blocks is multiple; The determination of the model constraints for the surface includes: Based on the preset size, preset shape, and preset arrangement of each calibration block, the surface equation of the surface is determined as the model constraint condition of the surface; The determination of the extrinsic parameters that match the local point cloud data of each surface with the model constraints of each surface, as the target extrinsic parameters of the profilometer camera, includes: Based on the model constraints of each surface, the correspondence between model constraint error and extrinsic parameters is determined, wherein the model constraint error is: the error of the local point cloud data of each surface relative to the model constraints of each surface when the extrinsic parameters of the profilometer camera are the extrinsic parameters corresponding to the model constraint error; Based on the correspondence, the extrinsic parameter corresponding to the minimum value of the model constraint error is determined and used as the target extrinsic parameter of the profilometer camera.

2. The method according to claim 1, characterized in that, The step of determining local point cloud data belonging to each surface of the calibration block from the three-dimensional point cloud data includes: The point cloud data in the three-dimensional point cloud data is fitted to obtain the features of the fitting curve to which each point cloud data belongs, which are used as the surface features of the point cloud data. For each surface, according to the preset correspondence between surface features and surfaces, the point cloud data corresponding to the surface features is determined from the three-dimensional point cloud data, and used as the local point cloud data of the surface.

3. The method according to claim 2, characterized in that, The fitting of point cloud data in the three-dimensional point cloud data includes: Based on the preset shape of the calibration block, determine the type of each contour line of the calibration block; The point cloud data in the three-dimensional point cloud data is fitted to a fitting curve of the type mentioned above, and the characteristics of the fitting curve to which each point cloud data belongs are obtained, which are used as the surface features of the point cloud data.

4. The method according to claim 1, characterized in that, The determination of the model constraints for the surface includes: The surface equation of the surface is determined based on the preset size and preset shape of the surface, and serves as the model constraint condition for the surface.

5. The method according to claim 1, characterized in that, The acquisition of 3D point cloud data obtained from the profilometer camera's calibration block includes: Acquire multiple frames of 3D point cloud data obtained by the profilometer camera from calibration blocks located at different positions; Determining local point cloud data belonging to the surface from the three-dimensional point cloud data includes: Local point cloud data belonging to the surface are determined from each frame of 3D point cloud data.

6. A calibration device for a profilometer camera, characterized in that, The device includes: The acquisition module is used to acquire 3D point cloud data obtained by the profilometer camera from the calibration block. The first determining module is used to determine the model constraints of each surface of the calibration block; and to determine the local point cloud data belonging to the surface from the three-dimensional point cloud data. The second determining module is used to determine the extrinsic parameters that match the local point cloud data of each surface with the model constraint conditions of each surface, and use them as the target extrinsic parameters of the profilometer camera. The number of calibration blocks is multiple, and the shape of the calibration blocks is a cone, a spherical cap, a trapezoidal frustum, or a crater; the first determining module is further used for: Based on the preset size, preset shape, and preset arrangement of each calibration block, the surface equation of the surface is determined as the model constraint condition of the surface; The second determining module is specifically used for: Based on the model constraints of each surface, the correspondence between model constraint error and extrinsic parameters is determined, wherein the model constraint error is: the error of the local point cloud data of each surface relative to the model constraints of each surface when the extrinsic parameters of the profilometer camera are the extrinsic parameters corresponding to the model constraint error; Based on the correspondence, the extrinsic parameter corresponding to the minimum value of the model constraint error is determined and used as the target extrinsic parameter of the profilometer camera; The first determining module is specifically used for: The point cloud data in the three-dimensional point cloud data is fitted to obtain the features of the fitting curve to which each point cloud data belongs, which are used as the surface features of the point cloud data. For each surface, according to the preset correspondence between surface features and surfaces, the point cloud data corresponding to the surface features is determined from the three-dimensional point cloud data, and used as the local point cloud data of the surface. The fitting of point cloud data in the three-dimensional point cloud data includes: Based on the preset shape of the calibration block, determine the type of each contour line of the calibration block; The point cloud data in the three-dimensional point cloud data is fitted to the fitting curve of the type, and the features of the fitting curve to which each point cloud data belongs are obtained, which are used as the surface features of the point cloud data. The first determining module is further configured to: Based on the preset size and preset shape of the surface, the surface equation of the surface is determined as the model constraint condition of the surface; The acquisition module is used to acquire multiple frames of 3D point cloud data obtained by the profilometer camera from calibration blocks located at different positions. The first determining module is further configured to: Local point cloud data belonging to the surface are determined from each frame of 3D point cloud data.

7. An electronic device, characterized in that, The electronic device includes: Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the steps of the method described in any one of claims 1-5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the method described in any one of claims 1-5.

Images