Calibration method of contourgraph, computer program product and equipment

Abstract

The embodiment of the invention provides a contourgraph calibration method, a computer program product and equipment. In the process that the mobile platform drives the plane calibration object to move, the plane calibration object is scanned by using the contourgraph, and three-dimensional coordinates of feature points in the plane calibration object in a contourgraph coordinate system are determined based on data acquired by the contourgraph. And determining the conversion relation between the contourgraph coordinate system and the mobile platform coordinate system based on the three-dimensional coordinates of the feature points in the contourgraph coordinate system and the relative position relation of the feature points in the mobile platform coordinate system. Compared with a three-dimensional calibration object, the plane calibration object is more convenient to process and manufacture, the weight of the calibration object can be greatly reduced, and the plane calibration object is convenient to use.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of machine vision, in particular to a profile instrument calibration method, a computer program product and an apparatus. BACKGROUND

[0002] The profile instrument can be used to measure the linear shape and cross-sectional profile shape of various mechanical parts. Generally, when a laser profile instrument is used to measure an object to be measured, the object to be measured is placed on a moving platform, and the moving platform drives the object to be measured to move through rotation or translation. The profile instrument continuously collects point clouds of each profile cross section of the object to be measured during the movement of the object to be measured, and then splices the collected point clouds of multiple profile cross sections, so that the overall profile information of the object to be measured can be obtained. In the above process, since the point clouds of multiple profile cross sections need to be spliced, the extrinsic parameters between the profile instrument and the moving platform need to be calibrated. In the related art, a three-dimensional calibration object is usually used to calibrate the profile instrument. When the field of view of the profile instrument is very large, the size of the three-dimensional calibration object also needs to be very large, which is difficult to process and transport. SUMMARY

[0003] Therefore, the present application provides a profile instrument calibration method, a computer program product and an apparatus.

[0004] According to a first aspect of the present application, a profile instrument calibration method is provided, which comprises:

[0005] During the movement of the moving platform, the profile instrument to be calibrated is used to scan the planar calibration object to obtain a depth image and a brightness image corresponding to the planar calibration object, wherein the planar calibration object comprises a plurality of feature points, the relative positional relationship of the plurality of feature points in the moving platform coordinate system is known, the Y axis of the moving platform coordinate system is parallel to the scanning direction of the profile instrument scanning the planar calibration object, and the X axis of the moving platform coordinate system coincides with the plane on which the planar calibration object is located;

[0006] The plurality of feature points are extracted from the brightness image, and the three-dimensional coordinates of the plurality of feature points in the profile instrument coordinate system are determined based on the depth image, wherein the X axis and the Z axis of the profile instrument coordinate system are located on the laser plane of the profile instrument, and the Y axis of the profile instrument coordinate system is perpendicular to the laser plane;

[0007] Based on the three-dimensional coordinates of the plurality of feature points in the profile instrument coordinate system and the relative positional relationship of the plurality of feature points in the moving platform coordinate system, the conversion relationship between the profile instrument coordinate system and the moving platform coordinate system is determined.

[0008] According to a second aspect of this application, an electronic device is provided, the electronic device including a processor, a memory, and a computer program stored in the memory that is executable by the processor, wherein the processor executes the computer program to implement the method mentioned in the first aspect above.

[0009] According to a third aspect of this application, a computer program product is provided, the computer program product comprising a computer program that, when executed, implements the method mentioned in the first aspect above.

[0010] According to a fourth aspect of this application, a computer-readable storage medium is provided, on which a computer program is stored, which, when executed, implements the method mentioned in the first aspect above.

[0011] This application provides a planar calibration object for calibrating the transformation relationship (i.e., relative pose relationship) between the profilometer coordinate system and the moving platform coordinate system, and provides a method for calibrating the transformation relationship between the two coordinate systems using this planar calibration object. The planar calibration object may include multiple feature points, the relative positional relationship of which in the moving platform coordinate system is known, such as the known distance between the feature points or the known spatial positional relationship between the lines formed by the feature points. When scanning the planar calibration object using a profilometer, the three-dimensional coordinates of these feature points in the profilometer coordinate system can be determined based on the data collected by the profilometer. Then, based on the three-dimensional coordinates of each feature point in the profilometer coordinate system and their relative positional relationship in the moving platform coordinate system, the transformation relationship between the profilometer coordinate system and the moving platform coordinate system can be determined. Compared to a three-dimensional calibration object, the planar calibration object is easier to process and manufacture, and can significantly reduce the weight of the calibration object, making it more convenient to use.

[0012] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this specification. Attached Figure Description

[0013] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0014] Figure 1 This is a schematic diagram of a profilometer scanning the object to be inspected.

[0015] Figure 2 This is a schematic diagram of the profilometer coordinate system according to an embodiment of this application.

[0016] Figure 3 is a flow chart of a profile gauge calibration method according to an embodiment of the present application.

[0017] Figure 4 is a schematic diagram of constructing a feature vector according to an embodiment of the present application.

[0018] Figure 5 is a schematic diagram of a planar calibration object according to an embodiment of the present application.

[0019] Figure 6 is a schematic diagram of constructing a feature vector according to another embodiment of the present application.

[0020] Figure 7 is a schematic diagram of a logic structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0021] The technical solutions in the embodiments of the present application will be apparently and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all the other embodiments obtained by a person of ordinary skill in the art without any creative work fall within the protection scope of the present application.

[0022] As shown in Figure 1 , when a profile gauge measures a measured object, the laser emitted by the profile gauge can form a laser plane. The laser plane can obtain a profile cross-section point cloud of the measured object after each scanning of the measured object. If the profile gauge continuously scans the measured object to obtain point clouds of multiple profile cross-sections, and the point clouds of the multiple profile cross-sections are spliced, the surface point cloud of the measured object can be obtained.

[0023] The direction of the movement of the profile gauge relative to the measured object is called a scanning direction. In theory, if the scanning direction is perpendicular to the laser plane, and the relative movement distance (i.e., a scanning step) of the profile gauge relative to the measured object is equal in adjacent two times of scanning, the point cloud of the measured object in the profile gauge coordinate system can be expressed as:

[0024]

[0025] Y S =id·step

[0026] wherein id represents the id-th scanning of the profile gauge to the object, and step is the scanning step.

[0027] In actual scene arrangement, in order to realize the relative movement of the profilometer and the object to be measured, the object to be measured is usually placed on a moving platform in the measurement process, and the object to be measured is driven to move by the moving platform to realize the relative movement of the two. Since it is usually impossible to guarantee that the laser plane projected by the profilometer and the scanning direction are absolutely perpendicular when the profilometer and the moving platform are erected, that is, between two adjacent scans, the laser plane not only exists Y direction offset, but also exists X and Z direction offset. If the point clouds of each profile picture in the profilometer coordinate system are still directly spliced according to the Y axis, and the surface point cloud of the object to be measured is reconstructed, the point cloud obtained by reconstruction will be deformed, and the larger the angle between the scanning direction and the normal vector of the laser plane, the more exaggerated the point cloud deformation will be. Therefore, before the profilometer scans the object to be measured, the relative pose relationship between the scanning direction and the laser plane needs to be calibrated. When the positions of the profilometer and the moving platform are fixed, that is, the relative pose relationship between the profilometer and the moving platform is fixed, the relative pose relationship between the scanning direction and the laser plane is also fixed. Therefore, two coordinate systems can be defined: the profilometer coordinate system and the moving platform coordinate system. The profilometer coordinate system is related to the position of the laser plane of the profilometer, and the moving platform coordinate system is related to the position of the moving platform. Usually, as shown in Figure 2 the X and Z planes of the profilometer coordinate system are located on the laser plane projected by the profilometer, the Y axis of the profilometer coordinate system is parallel to the laser plane, and the Y axis of the moving platform coordinate system is parallel to the scanning direction. In this case, the relative pose of the profilometer coordinate system and the moving platform coordinate system is the relative pose relationship between the scanning direction and the laser plane. By calibrating the conversion relationship between the profilometer coordinate system and the moving platform coordinate system, the point cloud data of each profile picture collected by the profilometer can be spliced based on the conversion relationship to obtain the surface point cloud of the object to be detected.

[0028] Usually, when calibrating the relative pose relationship between the profilometer coordinate system and the moving platform coordinate system, a calibration object needs to be used. In related technologies, a three-dimensional calibration object (such as a small ball or a three-dimensional boss) is usually used to calibrate the profilometer. In the case that the scanning field of view of the profilometer is relatively large, the size of the three-dimensional calibration object also needs to be relatively large, so that the three-dimensional calibration object has a large volume and weight, is difficult to process and transport, and is not convenient to use.

[0029] To overcome the above problems, the embodiment of the present application designs a planar calibration object for calibrating the conversion relationship (i.e. relative pose relationship) between the coordinate system of the profilometer and the coordinate system of the mobile platform, and provides a method for calibrating the conversion relationship between the two coordinate systems by using the planar calibration object. The planar calibration object can include a plurality of feature points, and the relative position relationship of the plurality of feature points in the mobile platform coordinate system is known, such as the distance between the plurality of feature points, or the spatial position relationship between the straight lines formed by the plurality of feature points, etc. When the profilometer is used to scan the planar calibration object, the three-dimensional coordinates of the feature points in the profilometer coordinate system can be determined based on the data collected by the profilometer, and then the conversion relationship between the profilometer coordinate system and the mobile platform coordinate system can be determined based on the three-dimensional coordinates of the feature points in the profilometer coordinate system and the relative position relationship of the feature points in the mobile platform coordinate system. Compared with the three-dimensional calibration object, the planar calibration object is more convenient to process and manufacture, and can greatly reduce the weight of the calibration object, which is convenient to use.

[0030] As shown in Figure 3 The profilometer calibration method provided by the embodiment of the present application can include the following steps:

[0031] S302, in the process of moving the planar calibration object by the mobile platform, the profilometer to be calibrated is used to scan the planar calibration object, and the depth image and the brightness image corresponding to the planar calibration object are obtained, wherein the planar calibration object includes a plurality of feature points, the relative position relationship of the plurality of feature points in the mobile platform coordinate system is known, the Y axis of the mobile platform coordinate system is parallel to the moving direction of the planar calibration object, and the X axis of the mobile platform coordinate system coincides with the plane where the planar calibration object is located;

[0032] To calibrate the profilometer, the embodiment of the present application designs a two-dimensional planar calibration object, which includes a plurality of feature points, and the relative position relationship of the plurality of feature points in the mobile platform coordinate system is known. The known relative position relationship can be that the distance between the feature points is known, or the spatial position relationship between the straight lines formed by different feature points is known, such as that the straight lines are parallel to each other, perpendicular to each other, or the included angle between the straight lines is a fixed included angle, etc. Since the mobile platform coordinate system is a rectangular coordinate system, the size and shape of the planar calibration object will not be deformed in the mobile platform coordinate system, i.e. the size known line segment in the planar calibration object or the straight line with known spatial position relationship will not change in the mobile platform coordinate system. Therefore, as long as the relative position relationship of the feature points in the three-dimensional space is known, the relative position relationship of the feature points in the mobile platform coordinate system is also known.

[0033] In some scenarios, the surface of the planar calibration object can include a dot matrix, distances between adjacent dots in the dot matrix are known, and the feature points can be one or more dots in the dot matrix.

[0034] In some scenarios, the surface of the planar calibration object can include a checkerboard, a side length of each square in the checkerboard is known, and the feature points can be one or more corner points of the squares.

[0035] In the embodiments of the present application, the Y-axis of the mobile platform coordinate system is parallel to the moving direction of the planar calibration object, and the X-axis of the mobile platform coordinate system is coincident with the plane on which the planar calibration object is located. Of course, if the pose of the planar calibration object is constantly changed during the measurement, and the profilometer is used to scan the planar calibration object multiple times, the plane on which the planar calibration object is located is coincident with the plane on which the planar calibration object is located at the first time of scanning.

[0036] In step S302, when calibrating the profilometer coordinate system and the mobile platform coordinate system, a pre-designed planar calibration object can be placed in the mobile platform. During the movement of the mobile platform driving the planar calibration object, the profilometer to be calibrated can be used to scan the planar calibration object to obtain a depth image and a brightness image corresponding to the planar calibration object. When the profilometer scans the object to be detected, the point cloud of the object to be detected, i.e., the depth image, can be obtained, and at the same time, the brightness image of the object to be detected can also be obtained. The brightness image can be a black and white image.

[0037] S304, extracting the plurality of feature points from the brightness image, and determining three-dimensional coordinates of the plurality of feature points in the profilometer coordinate system based on the depth image, wherein the X-axis and the Z-axis of the profilometer coordinate system are located on the laser plane of the profilometer, and the Y-axis of the profilometer coordinate system is perpendicular to the laser plane;

[0038] In step S304, after the profilometer acquires the depth image and the brightness image of the planar calibration object, the plurality of feature points can be extracted from the brightness image. The extraction of the feature points can use a general algorithm, such as the Harris corner detection algorithm, the SIFT algorithm, or the extraction of the feature points from the brightness image through deep learning. The specific algorithm can be selected based on the actual application scenario, and the embodiments of the present application do not make any limitation.

[0039] Since the pixel points in the brightness image and the pixel points in the depth image are one-to-one corresponding, after the feature points are extracted from the brightness image, the positions of the feature points in the depth image can be determined based on the corresponding relationship between the pixel points, and then the three-dimensional coordinates of the feature points in the profilometer coordinate system can be determined.

[0040] For example, the Y-axis of the mobile platform coordinate system is parallel to the moving direction of the planar calibration object, and the X-axis of the mobile platform coordinate system is coincident with the plane on which the planar calibration object is located. Figure 2As shown, in the embodiments of the present application, the X axis and the Z axis of the profilometer coordinate system are located on the laser plane of the profilometer, and the Y axis of the profilometer coordinate system is perpendicular to the laser plane.

[0041] In step S306, after the three-dimensional coordinates of the plurality of feature points in the profilometer coordinate system are determined, the relative positional relationship of the plurality of feature points in the profilometer coordinate system can be determined. Since the relative positional relationship of the plurality of feature points in the mobile platform coordinate system is known, the conversion relationship between the profilometer coordinate system and the mobile platform coordinate system can be solved.

[0042] In some embodiments, when the conversion relationship between the profilometer coordinate system and the mobile platform coordinate system is determined based on the three-dimensional coordinates of the plurality of feature points in the profilometer coordinate system and the relative positional relationship of the plurality of feature points in the mobile platform coordinate system, one or more feature vectors can be constructed based on the plurality of feature points, wherein the length of the vector representation of the constructed one or more feature vectors in the mobile platform coordinate system and / or the spatial positional relationship thereof satisfy a preset condition. For example, the length of the constructed one or more feature vectors is known, or the spatial positional relationship of the constructed one or more feature vectors is known, such as that the plurality of constructed feature vectors are parallel to each other, the plurality of constructed feature vectors are perpendicular to each other, or the spatial positional relationship of the constructed feature vector and a known plane (such as a plane formed by the coordinate axes of the profilometer coordinate system or the mobile platform coordinate system) is known, or the relative positional relationship of the constructed feature vector and the coordinate axes of the profilometer coordinate system or the mobile platform coordinate system is known, and the like.

[0043] Since the three-dimensional coordinates of the plurality of feature points in the profilometer coordinate system are known, the vector representation of the constructed feature vector in the profilometer coordinate system can be determined based on the three-dimensional coordinates of the plurality of feature points in the profilometer coordinate system. Then, based on the vector representation of the feature vector in the profilometer coordinate system and the preset condition that the vector representation of the feature vector in the mobile platform coordinate system needs to satisfy, the conversion relationship can be solved.

[0044] For example, it is assumed that a feature vector A is constructed, the length of the feature vector A in the mobile platform coordinate system is d, and the vector representation of the feature vector A in the profilometer coordinate system is (a1, b1, c1). It is assumed that the conversion relationship between the profilometer coordinate system and the mobile platform coordinate system is R, and the vector representation of the feature vector A in the mobile platform coordinate system is (a1, b1, c1) x R. Then, R can be solved by taking the length of (a1, b1, c1) x R as d as a constraint.

[0045] In some embodiments, when the preset condition based on the vector representation of the feature vectors in the coordinate system of the profiler and the vector representation of the feature vectors in the coordinate system of the mobile platform is satisfied, the conversion relationship is solved, the initial value of the conversion relationship can be determined based on the preset condition based on the vector representation of the feature vectors in the coordinate system of the profiler and the vector representation of the feature vectors in the coordinate system of the mobile platform, and then the constraint condition can be constructed based on the three-dimensional coordinates of the plurality of feature points in the coordinate system of the profiler, the relative positional relationship of the plurality of feature points in the coordinate system of the mobile platform, and the condition required to be satisfied by the coordinate axes of the coordinate system of the profiler and the coordinate system of the mobile platform. The optimization of the initial value is performed based on the constructed constraint condition to obtain the optimized value of the conversion relationship.

[0046] Since there is a certain error in the feature points extracted from the brightness image, that is, the feature points extracted from the brightness image may not be the true feature points, the vector representation of the feature vector A in the coordinate system of the mobile platform based on the feature points may not actually satisfy the above-mentioned preset condition, for example, the length of the vector may not be the original length, or the vector may not be perpendicular, and therefore, the conversion relationship obtained by taking the preset condition of the vector representation of the feature vector in the coordinate system of the mobile platform as a constraint may not be accurate. In order to obtain a more accurate conversion relationship, the constraint condition can be further constructed based on the three-dimensional coordinates of the plurality of feature points in the coordinate system of the profiler and the relative positional relationship of the plurality of feature points in the coordinate system of the mobile platform, and the condition required to be satisfied by the coordinate axes of the coordinate system of the profiler and the coordinate system of the mobile platform (for example, the X axis of the coordinate system of the mobile platform coincides with the plane on which the plane calibration object is located, the Y axis is perpendicular to the scanning direction, and the like). The initial conversion relationship is optimized to obtain a more accurate optimized value.

[0047] In some embodiments, the constructed feature vectors include a first vector and a second vector, and the vector representation of each of the first vector and the second vector in the coordinate system of the mobile platform is perpendicular to each other. For example, in some scenarios, the plurality of feature points include feature point 1, feature point 2, feature point 3, and feature point 4, a straight line 1 connected by feature point 4 and feature point 2, and a straight line 2 connected by feature point 3 and feature point 4 are perpendicular to each other, and therefore, a feature vector A can be constructed based on feature point 2 and feature point 2, a feature vector B can be constructed based on feature point 3 and feature point 4, and the feature vector A and the feature vector B are perpendicular in the coordinate system of the mobile platform.

[0048] For example, taking the feature points as dot matrix as an example, the straight line connected by the points in the row of the dot matrix is perpendicular to the straight line connected by the points in the column of the dot matrix, and therefore, a feature vector A can be constructed based on the feature points in the row of the dot matrix, a feature vector B can be constructed based on the feature points in the column of the dot matrix, and the feature vector A and the feature vector B are perpendicular in the coordinate system of the mobile platform.

[0049] For example, in some embodiments, the planar calibration object surface comprises a checkerboard, the plurality of feature points are the corner points in the checkerboard, and the first vector and the second vector are the vectors represented by two adjacent edges of any corner point in the checkerboard as the vertex. Since the two edges of a corner point in the checkerboard are perpendicular in the mobile platform coordinate system, a feature vector A and a feature vector B can be constructed based on the two edges of the corner point in the checkerboard, and the feature vector A and the feature vector B are perpendicular in the mobile platform coordinate system.

[0050] In some embodiments, the constructed feature vector comprises a third vector, the X component of the vector representation of the third vector in the profiler coordinate system is 0, and the length of the vector representation of the third vector in the mobile platform coordinate system is known. Considering that the conversion relationship between the profiler coordinate system and the mobile platform coordinate system comprises a rotation matrix R MS , R MS is a 3x3 matrix, and is assumed to be In order to facilitate the solution of the elements of each row and each column in the rotation matrix R MS , in some scenarios, a feature vector can be constructed, the X component of the vector representation P s of the feature vector in the profiler coordinate system is 0, and is assumed to be [0, Ys, Zs]. Therefore, the vector representation P M of the feature vector in the mobile platform coordinate system is R MS P s Since the X component of Ps is 0, it can eliminate part of the elements of R MS after multiplication with R MS , and since the length of the vector representation in the mobile platform coordinate system is known, the elements of R MS that are not eliminated can be solved.

[0051] In some embodiments, the starting point of the third vector is any corner point in the checkerboard in the brightness image, and the ending point of the third vector is the intersection point of the downward extension of the any corner point until it contacts another checkerboard grid edge. Since the scanning direction is not perpendicular to the laser plane, the squares in the checkerboard have been scanned into parallelograms. Therefore, a feature vector can be constructed based on the checkerboard in the brightness image, for example, as shown in Figure 4 , the upper left corner of the checkerboard grid can be selected as the starting point to go down until it contacts another checkerboard grid edge to form a feature vector. The length d of the vector in the system coordinate system is known, and the X component of the vector in the profiler coordinate system is 0.

[0052] In some embodiments, the constructed feature vector includes the normal vector of the plane containing the planar calibration object, and the vector representation of this normal vector in the moving platform coordinate system is perpendicular to the X-axis of the moving platform coordinate system. Since we define the X-axis of the moving platform coordinate system to coincide with the plane containing the planar calibration object, that is, the normal vector of the plane containing the planar calibration object is perpendicular to the X-axis of the moving platform coordinate system, we can construct a feature vector that includes the normal vector of the plane containing the planar calibration object. Using the perpendicularity of the vector representation of this normal vector in the moving platform coordinate system to the X-axis of the moving platform coordinate system as a constraint, we can solve the above transformation relationship.

[0053] For example, such as Figure 5 As shown, assume the surface of the planar calibration object includes a checkerboard pattern, with some squares covered by QR codes. The multiple feature points are the corner points of the squares covered by the QR codes. The position information of these feature points within the checkerboard pattern is obtained by recognizing the QR codes in the brightness image. Assume the transformation relationship between the profilometer coordinate system and the moving platform coordinate system includes a rotation matrix (translation matrix not considered) R. MS R MS Let it be a 3×3 matrix, assuming it is... To solve R MS The elements in the matrix can be used to construct the following feature vector:

[0054] (1) As Figure 4 As shown, the top left corner of a checkerboard square can be chosen as the starting point to move downwards until it touches another checkerboard square, thus forming a feature vector. The magnitude d of this vector in the system coordinate system is known, and the X component of this vector in the profilometer coordinate system is 0.

[0055] The vector representation of this eigenvector in the profilometer coordinate system is P. s The X component is 0, assumed to be [0, Ys, Zs]. Therefore, the vector representation of this eigenvector in the moving platform coordinate system is P. M For P s ×R MS Since the X component of Ps is 0, its relationship with R... MS After multiplication, R can be eliminated. MS Some elements, for example, Since the magnitude of the vector in the coordinates of the moving platform is known, R can be solved. MS The element in the 2nd row and 3rd column.

[0056] (2) Figure 6 As shown, two vectors can be constructed using the two adjacent edges of the top-left vertex of the chessboard. The vector representations of the two vectors in the profilometer coordinate system are as follows:

[0057]

[0058] The vector representation of the two vectors in the mobile platform coordinate system is as follows:

[0059]

[0060] Since the two adjacent sides of the checkerboard in the system coordinate system are perpendicular, should be perpendicular to , that is, r 21 can be solved. Thus, r 21 , r 23 can be determined, and r 22 can be solved. That is, R MS can be determined. The second row of all elements.

[0061] (3) Construct the normal vector of the checkerboard plane, the X-axis of the mobile platform coordinate system is perpendicular to the normal vector, and the Y-axis of the mobile platform coordinate system is perpendicular to the normal vector. Based on this constraint, the elements in the matrix can be further solved.

[0062] In some embodiments, the constraint condition can include one or more of the following:

[0063] (1) Minimize the difference between the estimated three-dimensional coordinates of each feature point and the measured three-dimensional coordinates, wherein the estimated three-dimensional coordinates are determined based on the relative position relationship of the plurality of feature points in the mobile platform coordinate system and the conversion relationship, and the measured three-dimensional coordinates are the three-dimensional coordinates measured by the profiler. The measured three-dimensional coordinates can be determined based on the brightness image and the depth image collected by the profiler. In some scenarios, based on the conversion relationship between the profiler coordinate system and the checkerboard coordinate system, and the relative position relationship of the plurality of feature points in the mobile platform coordinate system, the three-dimensional coordinates of the plurality of feature points in the profiler coordinate system can be solved, which are referred to as estimated three-dimensional coordinates. The more accurate the conversion relationship is, the closer the estimated three-dimensional coordinates are to the measured three-dimensional coordinates. Therefore, the difference between the two coordinates can be minimized as a constraint to optimize the initial value of the conversion relationship.

[0064] (2) The X-axis of the mobile platform coordinate system is perpendicular to the normal vector of the plane calibration object, and the Y-axis of the mobile platform coordinate system is perpendicular to the normal vector. Since we define the X-axis of the mobile platform coordinate system to coincide with the plane of the plane calibration object, the X-axis of the mobile platform coordinate system is perpendicular to the normal vector of the plane calibration object, and the mobile platform coordinate system is a rectangular coordinate system, so the X-axis of the mobile platform coordinate system is perpendicular to the Y-axis of the mobile platform coordinate system. The above condition can be used as a constraint condition for optimizing the above conversion relationship to optimize the initial value of the conversion relationship.

[0065] (3) The Y-axis of the mobile platform coordinate system is perpendicular to the normal vector of the XOZ plane of the profiler coordinate system. In some embodiments, when the planar calibration object is scanned by the profiler, the planar calibration object can be placed in one fixed posture and scanned once. Since the planar calibration object is a two-dimensional plane, the single profile obtained by scanning is a profile line instead of a profile surface. At this time, the constraint on the conversion relationship (rotation matrix R) is insufficient. The rotation angle of the rotation matrix R around the single profile straight line can be arbitrary. The present constraint is to limit this degree of freedom. Therefore, the rotation matrix R can be prohibited from rotating around the direction vector of the single profile straight line. This can also be converted into the mobile platform coordinate system Y-axis being perpendicular to the normal vector of the single profile straight line. Since the straight line only has a normal vector concept in the plane, the profile straight line normal vector here is the normal vector of the XOZ two-dimensional plane of the profiler coordinate system.

[0066] Of course, if the planar calibration object is placed in different postures and then scanned by the profiler multiple times, the conversion relationship can be solved by using the data obtained by multiple scanning without adding constraint (3).

[0067] Since the planar calibration object is two-dimensional, it lacks one dimension of information compared to a three-dimensional calibration object. In order to make up for the missing dimension of information, the planar calibration object can be placed in different postures (for example, one end of the planar calibration object can be raised, etc.) when it is scanned by the profiler. Then the planar calibration object is scanned by the profiler multiple times, and the data obtained by multiple scanning is used to solve the above conversion relationship. For example, in some embodiments, when the planar calibration object is scanned by the profiler to be calibrated during the movement of the mobile platform driving the planar calibration object, the planar calibration object can be scanned by the profiler to be calibrated multiple times to obtain multiple depth images and multiple brightness images. In each scanning process, the relative pose of the planar calibration object and the profiler is different. When constructing constraints in three-dimensional coordinates of feature points, the measured three-dimensional coordinates of each feature point can include multiple groups. Each group of measured three-dimensional coordinates is determined based on the depth image and the brightness image obtained by the profiler scanning the planar calibration object once.

[0068] In some embodiments, as Figure 5As shown, the surface of the planar calibration object is a checkerboard, and part of the squares in the checkerboard are covered by a two-dimensional code. The plurality of feature points are the corner points of the squares covered by the two-dimensional code, and the position information of the plurality of feature points in the checkerboard is obtained by identifying the two-dimensional code in the brightness image. In the scenario where the surface of the planar calibration object is a checkerboard, since the checkerboard is a black-and-white grid, it is impossible to distinguish which corner point of the square is. In order to facilitate the determination of the position of the extracted feature points in the entire checkerboard, a two-dimensional code can be arranged in the checkerboard, and the feature points can be the corner points of the squares covered by the two-dimensional code. The two-dimensional code can carry the position information of the square in the checkerboard, for example, it can be the square in the mth row and the nth column. The position information of the feature points in the checkerboard can be obtained by identifying the two-dimensional code in the brightness image collected by the contour instrument.

[0069] In some embodiments, the contour instruments to be calibrated can include multiple. After determining the conversion relationship between the contour instrument coordinate system of each contour instrument and the mobile platform coordinate system, the relative poses between the multiple contour instruments can be determined based on the conversion relationship. By determining the relative poses between the multiple contour instruments, in the scenario of simultaneously scanning an object by using multiple contour instruments, the point cloud data collected by different contour instruments can be spliced based on the relative poses.

[0070] In some embodiments, considering that the planar calibration object is two-dimensional, it lacks one dimension of information compared to a three-dimensional calibration object, in order to make up for the missing dimension of information and obtain more accurate calibration results, when the profilometer is used to scan the planar calibration object, the planar calibration object can be placed in different postures (for example, one end of the planar calibration object can be raised), and then the profilometer is used to scan the planar calibration object multiple times. For example, in some embodiments, when the profilometer to be calibrated is used to scan the planar calibration object during movement of the mobile platform, the profilometer to be calibrated can be used to scan the planar calibration object multiple times to obtain multiple depth images and multiple brightness images, where the relative poses of the planar calibration object and the profilometer are different in each scanning process. The three-dimensional coordinates of each feature point in the plurality of feature points in the planar calibration object in the profilometer coordinate system each include multiple groups, and one group of three-dimensional coordinates of each feature point in the profilometer coordinate is obtained by one scanning of the profilometer on the planar calibration object. When the conversion relationship between the profilometer coordinate system and the mobile platform coordinate system is determined based on the three-dimensional coordinates of the plurality of feature points in the profilometer coordinate system and the relative positional relationship of the plurality of feature points in the mobile platform coordinate system, the conversion relationship between the profilometer coordinate system and the mobile platform coordinate system can be determined based on the multiple groups of three-dimensional coordinates of each feature point in the plurality of feature points in the profilometer coordinate system and the relative positional relationship of the plurality of feature points in the mobile platform coordinate system. By placing the planar calibration object in different postures and then using the profilometer to scan the planar calibration object multiple times from different angles, the conversion relationship can be solved by using the data obtained by multiple scanning, which can make the determined conversion relationship more accurate.

[0071] The solutions of the above embodiments can be freely combined to obtain new solutions in the absence of conflicts. Due to the page limit, they are not listed one by one.

[0072] In addition, the disclosure embodiments also provide a computer program product, which includes a computer program, and the computer program is executed to implement the method mentioned in any of the above embodiments.

[0073] The disclosure embodiments also provide an electronic device, such as Figure 7 As shown in the figure, the electronic device 70 includes a processor 71, a memory 72, and a computer instruction stored on the memory 72, and the processor executes the computer instruction to implement the method mentioned in any of the above embodiments.

[0074] Correspondingly, the disclosure embodiments also provide a computer storage medium, which stores a program, and the program is executed by a processor to implement the method in any of the above embodiments.

[0075] The embodiments of the present specification can adopt the form of a computer program product implemented on one or more storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing program codes. The computer usable storage media includes permanent and non-permanent, removable and non-removable media, and can be realized by any method or technology. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to: phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic disk storage or other magnetic storage device, or any other non-transmission medium that can be used to store information that can be accessed by a computing device.

[0076] For the device embodiment, since it basically corresponds to the method embodiment, the relevant part can refer to the part of the method embodiment. The device embodiment described above is only illustrative, wherein the units described as separate components can or can not be physically separated, and the components displayed as units can or can not be physical units, that is, they can be located in one place, or can be distributed to multiple network units. According to actual needs, part or all of the modules can be selected to achieve the purpose of the present embodiment. Those skilled in the art can understand and implement without creative labor.

[0077] The user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation portal for user to choose authorization or refusal.

[0078] It is to be noted that, in the present text, relational terms such as first and second and the like can be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. The terms "comprises", "comprising", or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises... a" does not, without more constraints, exclude the existence of additional identical elements in the process, method, article, or apparatus that comprises the element.

[0079] The above detailed description of the method and device provided by the embodiments of the present application has been introduced in detail, the principle and implementation mode of the present application are described by applying specific examples in the present text, the above embodiment explanation is only for helping to understand the method of the present application and its core idea; meanwhile, for the general technical personnel in the field, according to the idea of the present application, there will be changes in the specific implementation mode and application range, and the above-mentioned description should not be understood as the limitation of the present application.

Claims

1. A profilometer calibration method, characterized in that, The method includes: During the movement of the planar calibration object by the mobile platform, the planar calibration object is scanned by the profilometer to be calibrated to obtain the depth image and brightness image corresponding to the planar calibration object. The planar calibration object includes multiple feature points, and the relative positional relationship of the multiple feature points in the coordinate system of the mobile platform is known. The Y-axis of the coordinate system of the mobile platform is parallel to the scanning direction of the profilometer scanning the planar calibration object, and the X-axis of the coordinate system of the mobile platform coincides with the plane where the planar calibration object is located. The plurality of feature points are extracted from the brightness image, and the three-dimensional coordinates of each feature point in the profilometer coordinate system are determined based on the depth image, wherein the X-axis and Z-axis of the profilometer coordinate system are located on the laser plane of the profilometer, and the Y-axis of the profilometer coordinate system is perpendicular to the laser plane. Based on the three-dimensional coordinates of each of the multiple feature points in the profilometer coordinate system and the relative positional relationship of the multiple feature points in the moving platform coordinate system, the transformation relationship between the profilometer coordinate system and the moving platform coordinate system is determined.

2. The method according to claim 1, characterized in that, The process of determining the transformation relationship between the profilometer coordinate system and the mobile platform coordinate system based on the three-dimensional coordinates of each of the multiple feature points in the profilometer coordinate system and the relative positional relationship of the multiple feature points in the mobile platform coordinate system includes: One or more feature vectors are constructed based on the multiple feature points, wherein the magnitude and / or spatial position relationship of the vector representation of the one or more feature vectors in the coordinate system of the mobile platform meet preset conditions; Based on the three-dimensional coordinates of the multiple feature points in the profilometer coordinate system, determine the vector representation of the one or more feature vectors in the profilometer coordinate system. The transformation relationship is obtained by solving based on the vector representations of the one or more feature vectors in the profilometer coordinate system and the preset conditions that the vector representations of the one or more feature vectors in the mobile platform coordinate system must satisfy.

3. The method according to claim 2, characterized in that, The one or more feature vectors include a first vector and a second vector, wherein the vector representations of the first vector and the second vector in the coordinate system of the mobile platform are perpendicular to each other; and / or The one or more feature vectors include a third vector, the X component of which is 0 in the profilometer coordinate system, and the magnitude of which is known in the moving platform coordinate system; and / or The one or more feature vectors include the normal vector of the plane on which the planar calibration object is located, and the vector representation of the normal vector in the moving platform coordinate system is perpendicular to the X-axis of the moving platform coordinate system.

4. The method according to claim 3, characterized in that, The surface of the planar calibration object is a checkerboard pattern, the plurality of feature points are corner points of the checkerboard pattern, and the first vector and the second vector are vectors represented by two adjacent edges of any corner point in the checkerboard pattern; and / or The starting point of the third vector is any corner point in the checkerboard pattern of the brightness image, and the ending point of the third vector is the intersection point where the third vector extends downward from the corner point until it intersects with another checkerboard square edge.

5. The method according to claim 2, characterized in that, The transformation relationship is obtained by solving based on the preset conditions that the vector representations of the one or more feature vectors in the profilometer coordinate system and the vector representations of the one or more feature vectors in the moving platform coordinate system must satisfy, including: Based on the vector representations of the one or more feature vectors in the profilometer coordinate system and the preset conditions that the vector representations of the one or more feature vectors in the moving platform coordinate system must satisfy, the initial value of the transformation relationship is obtained. Constraints are constructed based on the three-dimensional coordinates of each feature point in the profilometer coordinate system, the relative positional relationship of the feature points in the moving platform coordinate system, and the conditions that each coordinate axis of the profilometer coordinate system and the moving platform coordinate system must satisfy. The initial value is optimized using the aforementioned constraints to obtain the optimized value of the transformation relationship.

6. The method according to claim 5, characterized in that, The constraints include one or more of the following: The difference between the estimated 3D coordinates and the measured 3D coordinates of each feature point is minimized, wherein the estimated 3D coordinates are 3D coordinates determined based on the relative positional relationship of the multiple feature points in the coordinate system of the moving platform and the optimized value of the transformation relationship, and the measured 3D coordinates are 3D coordinates measured by the profilometer. The X-axis of the mobile platform coordinate system is perpendicular to the normal vector of the planar calibration object and also perpendicular to the Y-axis of the mobile platform coordinate system. The Y-axis of the mobile platform coordinate system is perpendicular to the normal vector of the XOZ plane of the profilometer coordinate system.

7. The method according to claim 6, characterized in that, During the movement of the planar calibration object by the mobile platform, the planar calibration object is scanned using a profilometer to be calibrated, including: The planar calibration object is scanned multiple times using a profilometer to be calibrated, resulting in multiple frames of depth images and multiple frames of brightness images. During at least two scans, the relative pose of the planar calibration object and the profilometer is different. The measured three-dimensional coordinates of each feature point include multiple sets, and each set of measured three-dimensional coordinates is obtained by the profilometer scanning the planar calibration object once.

8. The method according to claim 1, characterized in that, The surface of the planar calibration object is a checkerboard pattern, and some squares in the checkerboard pattern are covered by QR codes. The multiple feature points are the corner points of the squares covered by the QR codes. The position information of the multiple feature points in the checkerboard pattern is obtained by recognizing the QR codes in the brightness image. And / or, The profilometers to be calibrated include multiple ones. After determining the transformation relationship between the profilometer coordinate system and the mobile platform coordinate system of each profilometer, the relative poses between the multiple profilometers are determined based on the transformation relationship.

9. The method according to claim 1, characterized in that, The three-dimensional coordinates of each of the multiple feature points in the profilometer coordinate system include multiple sets. During the movement of the planar calibration object by the moving platform, the planar calibration object is scanned using the profilometer to be calibrated, including: The planar calibration object is scanned multiple times using a profilometer to be calibrated, resulting in multiple frames of depth images and multiple frames of brightness images. In at least two scanning processes, the relative pose of the planar calibration object and the profilometer is different. A set of three-dimensional coordinates of each feature point in the profilometer coordinate system is obtained by the profilometer scanning the planar calibration object once. The process of determining the transformation relationship between the profilometer coordinate system and the mobile platform coordinate system based on the three-dimensional coordinates of each of the multiple feature points in the profilometer coordinate system and the relative positional relationship of the multiple feature points in the mobile platform coordinate system includes: Based on the multiple sets of three-dimensional coordinates of each feature point in the profilometer coordinate system and the relative positional relationship of the multiple feature points in the moving platform coordinate system, the transformation relationship between the profilometer coordinate system and the moving platform coordinate system is determined.

10. A computer program product, characterized in that, The computer program product includes a computer program that, when executed, implements the method described in any one of claims 1-9.

11. An electronic device, characterized in that, The electronic device includes a processor, a memory, and computer instructions stored in the memory, wherein the processor executes the computer instructions to implement the method according to any one of claims 1-9.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed, implements the method as described in any one of claims 1-9.

Images