Camera tilt correction method based on step standard block

Abstract

The invention relates to the technical field of image data processing, and discloses a camera tilt correction method based on a step standard block, the method is applied to a scene in which a scanning device is adopted to scan a standard object, the scanning device comprises a 3D line scanning camera, a mobile platform and the step standard block, the 3D line scanning camera is obliquely installed, and at the moment, the step standard block is installed on the mobile platform. The section XOZ of the camera is not perpendicular to the moving direction of the step standard block on the moving platform. According to the camera tilt correction method based on the step standard block, a point cloud correction algorithm is realized by calculating the point cloud conversion relation between the coordinates of the characteristic holes of the standard block on the 3D image and the design theoretical coordinates of the standard block, a certain angle tilt can be allowed to exist in the installation process of the line scanning camera, and the precision of the line scanning camera is improved. Accurate point cloud data can also be obtained through correction; the method is simple and fast, and can be conveniently used for camera correction in various occasions.

Description

Technical Field

[0001] The present application relates to the technical field of image data processing, and in particular to a camera tilt correction method based on a step block. Background Art

[0002] 3D detection scenarios are complex. In actual use, if the image to be scanned is an XYZ point cloud in an orthogonal coordinate system, a 3D line scan camera has very high installation requirements, that is, the XOZ surface of the single section line must be absolutely perpendicular to the platform movement direction, which is often impossible to achieve in actual application scenarios. Even due to special installation requirements, such as less occlusion problems when the camera is tilted;

[0003] Due to the inevitable inclination between the XOZ plane of the single profile line and the platform movement direction or the detection requirement (such as reducing occlusion), the point cloud obtained by scanning is a non-orthogonal coordinate system. In the actual detection and measurement process, measurements are often performed based on an orthogonal coordinate system, which requires the scanning point cloud of a single camera to be corrected. Otherwise, the accuracy of the detected data will be insufficient, which will lead to a decrease in the accuracy of the measurement results. Summary of the invention

[0004] The purpose of the present application is to provide a camera tilt correction method based on step blocks to solve the technical problems raised in the above background.

[0005] To achieve the above-mentioned purpose, the present application discloses the following technical solutions: a camera tilt correction method based on a step block, the method is applied to a scene where a scanning device is used to scan a target object, wherein the scanning device comprises a 3D line scan camera, a mobile platform and a step block, and the 3D line scan camera is installed at an angle, at which time the camera section XOZ is not perpendicular to the movement direction of the step block on the mobile platform; the method specifically comprises the following steps:

[0006] a. Use a 3D line scan camera to scan the block to obtain a 3D image;

[0007] b. Extract the center coordinates of each characteristic hole in the block in the image coordinate system UVW (P u ,P v ), and map it on the fitting surface of the step corresponding to the characteristic hole to obtain the hole center height information value P w , get the 3D coordinates of the center of the characteristic hole in the image coordinate system UVW (P u ,P v ,P w );

[0008] c. Calculate the 3D coordinates of the characteristic holes of the block on the 3D image (P u ,P v ,P w) and the theoretical coordinates of the block design in the orthogonal coordinate system (Q x ,Q y ,Q z )’s point cloud transformation relationship;

[0009] d. Convert the point cloud image to be processed according to the point cloud conversion relationship, obtain the converted point cloud image, and use it as the correct point cloud image data.

[0010] As a preference, the center coordinates (P u ,P v ), including:

[0011] Setting the properties of the Blob of the marked block feature hole, finding and screening the Blob to locate the initial position in the 3D image;

[0012] Perform circle fitting of feature fitting, extract the center millimeter position of each feature hole, and obtain the center coordinates of each feature hole in the image coordinate system UVW (P u ,P v ).

[0013] Preferably, the attribute of the Blob includes area or diameter.

[0014] As a preferred embodiment, the mapping is performed on the fitting surface of the step corresponding to the characteristic hole to obtain the hole center height information value P w , get the 3D coordinates of the center of the characteristic hole in the image coordinate system UVW (P u ,P v ,P w ), including:

[0015] The step surface plane where the center of the characteristic hole is located is fitted by a fitting method to obtain the plane equation PlaneA: Ax+By+Cz+D=0;

[0016] The center coordinates (P u ,P v ) is mapped to the plane equation of the corresponding surface to obtain the hole center height information value P w : PW = -(A*Pu+B*Pv+D) / A;

[0017] Get the 3D coordinates of the center of all feature holes in the camera image in the image coordinate system UVW (P u ,P v ,P w ).

[0018] Preferably, the fitting method is a least squares method or a RANSAC method.

[0019] Preferably, the 3D coordinates (P u ,P v ,P w ) and the theoretical coordinates of the block design in the orthogonal coordinate system (Q x ,Q y ,Q z ), specifically including:

[0020] By 3D coordinates (P u ,P v ,P w ) and the theoretical coordinates of the block design in the orthogonal coordinate system (Q x ,Q y ,Q z ), input the initial value of the camera installation tilt angle, calculate R, T, α, θ through the optimization library, and get the following transformation matrix:

[0021]

[0022] Where R is the orthogonal unit rotation matrix, T is the three-dimensional translation vector, α is the rotation angle around the x-axis, θ is the rotation angle around the z-axis, and P x , P y and P z are the coordinate values ​​of the points on the converted block corresponding to the X-axis, Y-axis and Z-axis in the orthogonal coordinate system, and M is the matrix between the image coordinate system UVW and the orthogonal coordinate system XYZ.

[0023] Preferably, the optimized library is a g2o library or a ceres library.

[0024] Preferably, the expression of the matrix M is:

[0025]

[0026] Among them, K 1 =cosα*cosθ, K 2 =cosα*sinθ,K 3 =sinα.

[0027] Beneficial effect: The camera tilt correction method based on the step block of the present application realizes a point cloud correction algorithm by calculating the point cloud conversion relationship between the coordinates of the characteristic hole of the block on the 3D image and the theoretical coordinates of the block design. This method allows the line scan camera to be tilted at a certain angle during the installation process, and can also obtain accurate point cloud data through correction. This method is simple and fast, and can be conveniently used for camera correction in various occasions. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without paying any creative work.

[0029] Figure 1 A schematic diagram of a process flow of a camera tilt correction method based on a step block provided in an embodiment of the present application;

[0030] Figure 2 A schematic diagram of a normal installation of a camera provided in an embodiment of the present application;

[0031] Figure 3 A schematic diagram of a camera provided in an embodiment of the present application when the camera is rotated inwards around an angle X α;

[0032] Figure 4 This is a schematic diagram of a camera provided in an embodiment of the present application when the rotation angle α around X is outward-inward. DETAILED DESCRIPTION

[0033] The technical solutions in the embodiments of the present application are described clearly and completely below. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present application.

[0034] In this article, the term "comprising" is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of more restrictions, the elements defined by the sentence "comprising..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0035] Due to the inevitable angle problem during the installation process, or in some applications, the camera is installed at an angle to reduce occlusion or other requirements, resulting in the coordinate system of the image data obtained by scanning is not an orthogonal coordinate system, that is, the camera section XOZ plane is not perpendicular to the movement direction. Figure 1The camera tilt correction method based on the step block shown is applied to the scene where a scanning device is used to scan a target object, wherein the scanning device includes a 3D line scan camera, a mobile platform and a step block, and the 3D line scan camera is installed at an angle. At this time, the camera section XOZ is not perpendicular to the movement direction of the step block on the mobile platform, and the y direction in the collected image uses the setting sampling data of the movement direction, and the movement direction is not perpendicular to the XOZ plane, which causes the image data to be a non-orthogonal coordinate system problem, thereby causing specific errors in the measurement. Specifically, the method includes the following steps:

[0036] a. Use a 3D line scan camera to scan the block to obtain a 3D image;

[0037] b. Extract the center coordinates of each characteristic hole in the block in the image coordinate system UVW (P u ,P v ), and map it on the fitting surface of the step corresponding to the characteristic hole to obtain the hole center height information value P w , get the 3D coordinates of the center of the characteristic hole in the image coordinate system UVW (P u ,P v ,P w );

[0038] c. Calculate the 3D coordinates of the characteristic holes of the block on the 3D image (P u ,P v ,P w ) and the theoretical coordinates of the block design in the orthogonal coordinate system (Q x ,Q y ,Q z )’s point cloud transformation relationship;

[0039] d. Convert the point cloud image to be processed according to the point cloud conversion relationship, obtain the converted point cloud image, and use it as the correct point cloud image data.

[0040] The step block has a plurality of step surfaces, and the distance between the characteristic holes on each step surface is processed according to the design size. Among them, the center coordinates (P u ,P v ), including:

[0041] Setting the properties of the Blob of the block feature hole, the properties of the Blob may be information such as area, diameter, etc., and finding and filtering the Blob to locate the initial position in the 3D image;

[0042] Perform circle fitting of feature fitting, extract the center millimeter position of each feature hole, and obtain the center coordinates of each feature hole in the image coordinate system UVW (P u ,Pv ).

[0043] Furthermore, the mapping is performed on the fitting surface of the step corresponding to the characteristic hole to obtain the hole center height information value P w , get the 3D coordinates of the center of the characteristic hole in the image coordinate system UVW (P u ,P v ,P w ), including:

[0044] By using the least square method or RANSAC method, the step surface plane where the center of the characteristic hole is located is fitted to obtain the plane equation PlaneA: Ax+By+Cz+D=0;

[0045] Since the center of the hole lacks height information, the center coordinates (P u ,P v ) is mapped to the plane equation of the corresponding surface to obtain the hole center height information value P w : PW = -(A*Pu+B*Pv+D) / A;

[0046] Get the 3D coordinates of the center of all feature holes in the camera image in the image coordinate system UVW (P u ,P v ,P w ).

[0047] In addition, the 3D coordinates (P u ,P v ,P w ) and the theoretical coordinates of the block design in the orthogonal coordinate system (Q x ,Q y ,Q z ), specifically including:

[0048] By 3D coordinates (P u ,P v ,P w ) and the theoretical coordinates of the block design in the orthogonal coordinate system (Q x ,Q y ,Q z ), input the initial value of the camera installation tilt angle, calculate R, T, α, θ (or K1, K2, K3, where K1-3 are described later) through optimization libraries such as g2o library or ceres library, and get the following transformation matrix:

[0049]

[0050] Where R is the orthogonal unit rotation matrix, T is the three-dimensional translation vector, α is the rotation angle around the x-axis, θ is the rotation angle around the z-axis, and P x , P y and P z are the coordinate values ​​of the points on the converted block corresponding to the X-axis, Y-axis and Z-axis in the orthogonal coordinate system, and M is the matrix between the image coordinate system UVW and the orthogonal coordinate system XYZ.

[0051] In this embodiment, the expression of the matrix M is:

[0052]

[0053] Among them, K 1 =cosα*cosθ, K 2 =cosα*sinθ,K 3 =sinα. Regarding the acquisition of the matrix M, this embodiment is described in detail in the following content.

[0054] Assuming that at any point P in space, the image coordinate system UVW and the orthogonal coordinate system XYZ have such a relationship, the X axis and the U axis are coaxial, and the Z axis and the W axis are coaxial, then there is the following relationship:

[0055]

[0056] Unit vector for UVW coordinate system Decompose in the XYZ coordinate system, assuming that the components on each axis of XYZ are (K 1 , K 2 , K 3 ),Right now:

[0057]

[0058] K 1 2 +K 2 2 +K 3 2 =1 Formula 4

[0059] Substitute formula 3 into formula 2, and the X-axis and U-axis are coaxial, and the Z-axis and W-axis are coaxial, then Can be transformed into:

[0060]

[0061] Right now:

[0062] P x =P u +P v *K 1

[0063] P y =P v *K 2

[0064] P z =P w +P v *K 3

[0065] There is a matrix M from the image coordinate system UVW to the orthogonal coordinate system XYZ, written as a matrix, we have:

[0066]

[0067] Among them, for k 1 ,k 2 ,k 3 The problem can be further analyzed:

[0068] When XOZ is not perpendicular to the direction of motion, we can understand the problem as an angle relationship between the V axis of the image coordinate system and the Y axis of the orthogonal coordinate system, which are α for rotation around the X axis, β for rotation around the Y axis, and θ for rotation around the Z axis.

[0069] We can artificially define the rotation relationship as first doing β rotation and then doing other rotations. After β rotation, the image coordinate system is still an orthogonal coordinate system. The transformation at this time is RT rigid transformation; then the impact on the data is α rotation around x and θ rotation around z.

[0070] Next, we can transform it into the problem of finding the rotation angles of α and θ, and we can get (cosα*cosθ, cosα*sinθ, sinα)

[0071]

[0072] Initial value given problem: In general scenes, the camera is installed with a small rotation angle around the Z axis. Then, the initial value of θ around the Z axis can be 0. When the camera rotates around the X axis and the tilt is large α, we can use a protractor to measure this value to obtain the initial value α. Common K 1 ,K 2 ,K 3 The initial value of is defined as follows:

[0073] like Figure 2 As shown, when the camera is installed normally (i.e., it is not tilted), we can give the initial values ​​0, 1, 0;

[0074] like Figure 3 As shown, when the rotation angle α of the camera around the X-axis is inward, we can give the initial value as 0, cosα, sinα;

[0075] like Figure 4 As shown, when the rotation angle α around X is external and internal, the initial value is 0, cosα, -sinα.

[0076] When the calibration block is placed, the design coordinate system based on the calibration block and the orthogonal coordinate system we defined cannot coincide, and there is an RT matrix. Then the entire transformation relationship can be defined as:

[0077]

[0078] Among them, (P x ,P y ,P z ) is the coordinate of the characteristic point of the actual designed block, (P u ,P v ,P w ) is the coordinate of the block feature point in the image coordinate system. We can use optimization tools such as g2o or ceres to solve it according to the input K 1 ,K 2 ,K 3 , first calculate the RT matrix, and then use K 1 ,K 2 ,K 3 and RT matrix to obtain accurate α, θ or K 1 ,K 2 ,K 3 .

[0079] Based on the above, it can be seen that this method is based on calculating the coordinates of the characteristic holes of the block on the 3D image (P u ,P v ,P w ) and the block design theoretical coordinates (Q x ,Q y ,Q z ) point cloud conversion relationship, and realize a point cloud correction algorithm. It can allow the line scan camera to have a certain angle of inclination during the installation process, and can also obtain accurate point cloud data through correction; because this method is simple and fast, it can be conveniently used for camera correction in various occasions.

[0080] In the embodiments provided in the present application, it should be understood that the embodiments described herein can be implemented in hardware, software, firmware, middleware, code or any appropriate combination thereof. For hardware implementation, the processor can be implemented in one or more of the following units: application specific integrated circuit (ASIC), digital signal processor (DSP), digital signal processing device (DSPD), programmable logic device (PLD), field programmable gate array (FPGA), processor, controller, microcontroller, microprocessor, other electronic units designed to implement the functions described herein or their combination. For software implementation, part or all of the flow of the embodiment can be completed by instructing the relevant hardware through a computer program. When implemented, the above program can be stored in a computer-readable storage medium or transmitted as one or more instructions or codes on a computer-readable storage medium. Computer-readable storage media include computer storage media and communication media, wherein the communication medium includes any medium that is convenient for transmitting a computer program from one place to another. The storage medium can be any available medium that a computer can access. The computer-readable storage medium can include but is not limited to RAM, ROM, EEPROM, CD-ROM or other optical disk storage, disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of an instruction or data structure and can be accessed by a computer.

[0081] Finally, it should be noted that the above is only a preferred embodiment of the present application and is not intended to limit the present application. Although the present application has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A camera tilt correction method based on step blocks, characterized in that: The method is applied to a scene where a scanning device is used to scan a target object, wherein the scanning device includes a 3D line scan camera, a mobile platform and a step block, and the 3D line scan camera is installed at an angle. At this time, the camera section XOZ is not perpendicular to the movement direction of the step block on the mobile platform; the method specifically includes the following steps: a. Use a 3D line scan camera to scan the block to obtain a 3D image; b. Extract the center coordinates (Pu, Pv) of each characteristic hole of the standard block in the image coordinate system UVW, and map them on the fitting surface of the step corresponding to the characteristic hole to obtain the hole center height information value Pw, and obtain the 3D coordinates (Pu, Pv, Pw) of the center of the characteristic hole in the image coordinate system UVW; c. Calculate the point cloud conversion relationship between the 3D coordinates (Pu, Pv, Pw) of the characteristic hole of the block on the 3D image and the theoretical coordinates (Qx, Qy, Qz) of the block design in the orthogonal coordinate system; d. Convert the point cloud image to be processed according to the point cloud conversion relationship, obtain the converted point cloud image, and use it as the correct point cloud image data.

2. The camera tilt correction method based on step blocks according to claim 1, characterized in that: The extraction of the center coordinates (Pu, Pv) of each characteristic hole of the standard block in the image coordinate system UVW specifically includes: Setting the properties of the Blob of the marked block feature hole, finding and screening the Blob to locate the initial position in the 3D image; Perform circle fitting of feature fitting, extract the center millimeter position of each feature hole, and obtain the center coordinates (Pu, Pv) of each feature hole in the image coordinate system UVW.

3. The camera tilt correction method based on step blocks according to claim 2, characterized in that: The properties of the Blob include area or diameter.

4. The camera tilt correction method based on step blocks according to claim 1, characterized in that: The mapping on the fitting surface of the step corresponding to the characteristic hole, obtaining the hole center height information value Pw, and obtaining the 3D coordinates (Pu, Pv, Pw) of the center of the characteristic hole in the image coordinate system UVW specifically includes: The step surface plane where the center of the characteristic hole is located is fitted by a fitting method to obtain the plane equation PlaneA: Ax+By+Cz+D=0; The center coordinates (Pu, Pv) of each characteristic hole in the image coordinate system UVW are mapped to the plane equation of the corresponding surface to obtain the hole center height information value Pw: PW = -(A*Pu+B*Pv+D) / A; Get the 3D coordinates (Pu, Pv, Pw) of the center of all feature holes in the camera image in the image coordinate system UVW.

5. The camera tilt correction method based on step blocks according to claim 4, characterized in that: The fitting method is the least square method or the RANSAC method.

6. The camera tilt correction method based on step blocks according to claim 1, characterized in that: The point cloud conversion relationship between the 3D coordinates (Pu, Pv, Pw) of the characteristic hole of the calculation block on the 3D image and the theoretical coordinates (Qx, Qy, Qz) of the block design in the orthogonal coordinate system specifically includes: By designing the theoretical coordinates (Qx, Qy, Qz) of the 3D coordinates (Pu, Pv, Pw) on the 3D image and the subscript block in the orthogonal coordinate system, inputting the initial value of the tilt angle of the camera installation, and calculating R, T, α, θ through the optimization library, the following transformation matrix is ​​obtained: Where R is the orthogonal unit rotation matrix, T is the three-dimensional translation vector, α is the rotation angle around the x-axis, θ is the rotation angle around the z-axis, Px, Py and Pz are the coordinate values ​​of the points on the converted block corresponding to the x-axis, y-axis and z-axis in the orthogonal coordinate system respectively, and M is the matrix between the image coordinate system UVW and the orthogonal coordinate system XYZ.

7. The camera tilt correction method based on step blocks according to claim 6, characterized in that: The optimization library is a g2o library or a ceres library.

8. The camera tilt correction method based on step blocks according to claim 6, characterized in that: The expression of matrix M is: Among them, K1=cosα*cosθ, K2=cosα*sinθ, K3=sinα.

Images