Laser vision fusion measurement method and system

Abstract

The invention provides a laser vision fusion measurement method and system, and belongs to the technical field of measurement technology data processing. According to point cloud data obtained by a laser scanner and target point data obtained by photographing vision measurement, an intermediate coordinate system is established through a common target point measured twice; and calculating an optimal coordinate transformation matrix between the laser scanning coordinate system and the photogrammetry coordinate system by using an optimization iterative algorithm in the intermediate coordinate system. According to the optimal coordinate transformation matrix, the photogrammetry coordinate system and the target point coordinate thereof are mapped into the laser scanning coordinate system, registration fusion of the two coordinate systems is completed, laser-vision fusion measurement is achieved, the requirement for high-precision testing equipment is lowered, and industrial cost-reducing efficiency-increasing production development is promoted.

Description

Technical Field

[0001] The present invention belongs to the technical field of measurement technology data processing, and in particular relates to a laser vision fusion measurement method and system. Background Art

[0002] After the product is processed and manufactured, high-precision measuring instruments are needed to observe the product quality, especially the key point position and deformation information in the assembly process are extremely important, which is used to evaluate the product production quality and adjust the subsequent work arrangements such as installation and assembly according to the production quality. The commonly used high-precision measurement methods in the industrial field are laser scanning and photogrammetry. The laser scanning method uses high-speed laser scanning to quickly obtain data information such as the coordinates of each point on the surface of an object over a large area; photogrammetry takes multiple pictures and processes the pictures to obtain data information such as the relative position of the key points in the picture relative to a certain part of the object and the displacement of the key points.

[0003] Laser scanning can quickly obtain the surface morphology of an object over a large area with high resolution. Based on the high-density 3D point cloud data and the surface data information of the object, the 3D shape of the object can be quickly obtained and reconstructed for comparison with the design model. However, laser scanning cannot extract the relative position information of a key point on a surface; while photogrammetry can obtain the relative position of specific key feature points to the surface morphology of an object, but the accuracy of extracting the overall shape information of the object is not high.

[0004] Existing laser scanning and photogrammetry fusion methods are commonly used to fuse high-precision point cloud data from laser scanning with high-density point cloud data from photogrammetry to form higher-quality fused point cloud parameters in order to achieve better scanning results in static conditions. However, the relative position of key nodes and workpieces is more important. The laser scanning method cannot obtain high-precision key point positions and micro-deformation information. At the same time, photogrammetry results cannot obtain high-precision object surface morphology data, but only the relative position information of key points. Summary of the invention

[0005] In order to fuse the 3D point cloud data obtained by laser scanning technology and the 3D target point data obtained by photogrammetry, thereby reducing the demand for ultra-high precision equipment and achieving the purpose of reducing manufacturing costs, the present invention proposes a laser vision fusion measurement method and system, which reduces the demand for high-precision testing equipment and promotes the development of industrialized cost-reduction and efficiency-enhancing production. The technical solution is as follows:

[0006] In a first aspect, a laser vision fusion measurement method is provided, the method comprising:

[0007] Step 1: Use a laser scanning system to perform laser scanning measurement on the surface of the measurement object to obtain three-dimensional point cloud data in the laser scanner coordinate system, wherein the three-dimensional point cloud data includes point cloud information on the surface of the measurement object and three-dimensional coordinates of target points, and a plurality of target points are arranged on the surface of the measurement object;

[0008] Step 2: Filter and denoise the three-dimensional point cloud data to establish a point cloud database information reflecting the shape of the measured object, wherein the point cloud database information includes the denoised surface point cloud information of the measured object and the three-dimensional coordinates of the target point;

[0009] Step 3: Use a photogrammetry system to perform multi-angle photogrammetry on the target points on the surface of the measurement object, obtain the three-dimensional coordinates of each target point in the photogrammetry coordinate system, and establish a database information of the target points on the surface of the measurement object;

[0010] Step 4: For the morphology point cloud database information and the target point database information, use the common target point in the two measurements as the key point of coordinate transformation, establish an intermediate coordinate system, obtain the three-dimensional coordinate information in the two database information under the intermediate coordinate system, and realize the fusion measurement of laser scanning and vision.

[0011] Furthermore, before using the laser scanning system to perform laser scanning measurement on the surface of the measurement object, the method further includes: pasting a plurality of target points on the surface of the measurement object.

[0012] Among them, the spacing of the target points pasted on the surface of the measurement object is determined according to the measurement range of the laser scanning system, so that there are more than 3 target points in the measurement field at all times when the laser scanning system is measuring, which is convenient for subsequent coordinate conversion.

[0013] Among them, during the laser scanning measurement in step one, the distance and angle between the laser scanning system and the measured object are adjusted according to the size of the measured object, so that there are more than three target points in the measurement field at all times during the measurement, which is convenient for subsequent coordinate conversion.

[0014] Optionally, the filtering and denoising process in step 2 adopts a radius filtering algorithm to remove outliers and noise points in the scanning results to obtain the topographic point cloud database information of the measured object.

[0015] Among them, in step 3, during the photogrammetry, the camera rotates around the measured object in sequence for one circle to take multi-angle shots. Each time the image is taken, there are more than three target points, which is convenient for subsequent coordinate conversion.

[0016] Among them, step four specifically includes:

[0017] After determining the common target points in the two measurement results, three target points on the same surface are randomly selected as the construction elements of the coordinate system, and the intermediate coordinate system is established in the laser scanning coordinate system and the photogrammetry coordinate system respectively;

[0018] According to the coordinate transformation method, the three-dimensional position coordinates of all points in the morphology point cloud database information are transformed in the same scanning result, the laser scanning coordinate system and the photogrammetry coordinate system are transformed into the intermediate coordinate system, and the three-dimensional coordinate information in the two databases under the intermediate coordinate system is obtained.

[0019] Furthermore, after step 4, the method further includes:

[0020] Step 5: align and match the three-dimensional coordinate information of the common target points in the two databases in the intermediate coordinate system. According to the optimization iteration theory, the optimal coordinate transformation matrix between the coordinate systems corresponding to the information in the two databases is obtained, and the two coordinate systems are aligned and unified. The laser scanning coordinate system is used as the reference to realize the fusion measurement of laser scanning and vision. This can further improve the accuracy of fusion measurement.

[0021] In a second aspect, a laser vision fusion measurement system is provided, comprising: a laser scanning system, a photogrammetry system, and a processor.

[0022] The laser scanning system is used to perform laser scanning measurement on the surface of the measured object to obtain three-dimensional point cloud data in the laser scanner coordinate system;

[0023] The photogrammetry system is used to perform multi-angle photogrammetry of target points on the surface of the measured object to obtain the three-dimensional coordinates of each target point in the photogrammetry coordinate system;

[0024] Among other things, the processor is used to:

[0025] Perform filtering and denoising on the 3D point cloud data to establish a point cloud database information reflecting the shape of the measured object;

[0026] Establishing target point database information on the surface of the measurement object;

[0027] For the morphological point cloud database information and the target point database information, the common target point in the two measurements is used as the key point of coordinate transformation, an intermediate coordinate system is established, and the three-dimensional coordinate information in the two database information under the intermediate coordinate system is obtained.

[0028] Furthermore, the processor is also used to: align and match the three-dimensional coordinate information of the common target point in the two databases under the intermediate coordinate system, obtain the optimal coordinate transformation matrix between the coordinate systems corresponding to the two database information according to the optimization iteration theory, align the two coordinate systems, and use the laser scanning coordinate system as a reference to realize the fusion measurement of laser scanning and vision.

[0029] The present invention aims at the point cloud data obtained by the laser scanner and the target point data obtained by the photogrammetric measurement, and establishes an intermediate coordinate system through the common target point of the two measurements. An optimization iterative algorithm is used in the intermediate coordinate system to calculate the optimal coordinate transformation matrix between the laser scanning coordinate system and the photogrammetric coordinate system. According to the optimal coordinate transformation matrix, the photogrammetric coordinate system and its target point coordinates are mapped to the laser scanning coordinate system, and the registration and fusion of the two coordinate systems are completed, so as to realize the laser-vision fusion measurement. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 This is a flow chart of the laser vision fusion measurement method of the present invention;

[0031] Figure 2 This is a schematic diagram of coordinate fusion registration of the present invention;

[0032] Figure 3 It is a schematic diagram of the component surface point cloud and the three-dimensional coordinates of the target point in the laser scanning coordinate system of the present invention;

[0033] Figure 4 Schematic diagram of the three-dimensional coordinates of the target point in the photogrammetry coordinate system of the present invention;

[0034] Figure 5 It is a schematic diagram of the laser scanning point cloud and the photogrammetry target point in the intermediate coordinate system of the present invention;

[0035] Figure 6 It is a schematic diagram of the registration and fusion results of the laser scanning coordinate system and the photogrammetry coordinate system of the present invention. DETAILED DESCRIPTION

[0036] The present invention is further described in detail below through specific implementation modes and drawings.

[0037] The present invention aims at two different measurement methods, laser scanning and photogrammetry, takes into account the differences in coordinate system expression and measurement output of different measurement devices, and uses an optimized iterative method to achieve the registration and fusion of two measurement coordinate systems.

[0038] An embodiment of the present invention provides a laser vision fusion measurement method, which is implemented based on the optimization iteration theory, see Figure 1 and Figure 2 , the method comprises the following steps:

[0039] Step 1, pasting multiple target points on the surface of the measurement object;

[0040] The target points attached to the surface of the measurement object can be arranged at intervals according to the measurement range of the laser scanning system to ensure that there are more than three target points in the measurement field during the measurement by the laser scanning system, which is convenient for subsequent coordinate conversion. The measurement object in the embodiment of the present invention can be various tooling.

[0041] Step 2: Use a laser scanning system to perform laser scanning measurement on the surface of the measurement object to obtain three-dimensional point cloud data in the laser scanner coordinate system, and the three-dimensional point cloud data includes point cloud information on the surface of the measurement object and three-dimensional coordinates of the target point;

[0042] During laser scanning measurement, the distance and angle between the laser scanning system and the measured object need to be adjusted according to the size of the measured object to ensure that there are more than three target points in the measurement field at all times, which is convenient for subsequent coordinate conversion.

[0043] Step 3: Perform filtering and denoising on the 3D point cloud data from step 2 to establish Figure 3 The morphology point cloud database information reflecting the measured object shown includes the denoised surface point cloud information of the measured object and the three-dimensional coordinates of the target point;

[0044] The filtering and denoising process in this step can use a radius filtering algorithm to remove outliers and noise points in the scanning results and obtain the morphological point cloud database information of the measured object.

[0045] The radius filtering algorithm can be expressed as:

[0046] N i ≤N x

[0047] Among them, N i Represents the number of data points whose distance from point i is less than R; R represents the filtering threshold; N x Represents the noise point evaluation standard. That is, when the number of data points with a distance less than R from point i is small, it can be considered that the data points near the point are sparse, and the point is considered to be an outlier or noise point in the database, and should be discarded and the data point deleted.

[0048] Step 4: Use the photogrammetry system to perform multi-angle photogrammetry on the target points on the surface of the measured object to obtain Figure 4 The three-dimensional coordinates of each target point in the photogrammetric coordinate system are shown to establish the target point database information on the surface of the measurement object;

[0049] During photogrammetry, it is necessary to rotate around the measured object in sequence to take photos at multiple angles. Each image taken must contain more than three target points to facilitate subsequent coordinate conversion.

[0050] Step 5: For the topographic point cloud database information obtained by laser scanning (step 3) and the target point database information obtained by photogrammetry (step 4), the common target point in the two measurements is used as the key point of coordinate transformation to establish an intermediate coordinate system, as shown in the following figure: Figure 5The three-dimensional coordinate information in the two database information (i.e., the morphology point cloud database information and the target point database information) in the intermediate coordinate system shown realizes the fusion measurement of laser scanning and vision;

[0051] The specific process of establishing the intermediate coordinate system using the common target point in the two measurements as the key point of coordinate transformation is as follows:

[0052] 1. After determining the common target points in the two measurement results, arbitrarily select three target points on the same surface as the construction elements of the coordinate system, and establish the intermediate coordinate system in the laser scanning coordinate system and the photogrammetry coordinate system respectively;

[0053] 2. According to the coordinate transformation method, the three-dimensional position coordinates of all points in the morphology point cloud database information in the same scanning result are transformed, the laser scanning coordinate system and the photogrammetry coordinate system are transformed into the intermediate coordinate system, and the three-dimensional coordinate information in the two databases under the intermediate coordinate system is obtained.

[0054] The relationship before and after the coordinate transformation can be expressed as: P'=R*P+t,

[0055] Among them, P represents the original coordinate system before the transformation, P' represents the intermediate coordinate system after the transformation, R is the rotation matrix, t is the translation matrix, and the specific forms of R and t are as follows:

[0056]

[0057] Among them, α, β, and γ represent the rotation angles around the X, Y, and Z coordinate axes respectively, and t x ,t y ,t z They represent the translation distances along the X, Y, and Z coordinate axes respectively. The transformation matrix in the laser scanning system is denoted as H1, and the transformation matrix in the photogrammetry system is denoted as H2.

[0058] In order to further improve the accuracy of fusion measurement, another embodiment of the present invention also provides a laser vision fusion measurement method, see Figure 1 The method is consistent with the contents of steps 1 to 5 in the above embodiment. Further, the method further includes:

[0059] Step 6: align and match the three-dimensional coordinate information of the common target points in the two databases under the intermediate coordinate system. According to the optimization iteration theory, the optimal coordinate transformation matrix between the corresponding coordinate systems of the two database information (i.e., the morphological point cloud database information and the target point database information) is obtained, and the two coordinate systems are aligned and unified, with the laser scanning coordinate system as the reference, to achieve the following: Figure 6 Fusion measurement of laser scanning and vision shown.

[0060] The coordinate transformation relationship between the laser scanning coordinate system and the photogrammetry coordinate system can be expressed as:

[0061]

[0062] in, It is expressed as the three-dimensional coordinates of the i-th point in the intermediate coordinate system under the laser scanning method; It is expressed as the three-dimensional coordinates of the i-th point in the intermediate coordinate system under photogrammetry. R is the rotation matrix and t is the translation matrix. Selecting the same three points to establish two coordinate systems, it can be considered that the two coordinate systems are completely consistent in an ideal state. Therefore, at this time, the R matrix can be considered as a unit matrix, and t is considered to be zero. However, due to the existence of scanning noise and matching errors, the above formula may not be accurate.

[0063] In one achievable manner, step six may specifically include the following sub-steps:

[0064] 1. Accumulate the coordinate transformation errors to obtain the overall coordinate error information of the scanned workpiece. The details are as follows:

[0065] 1.1. First calculate the distance between coordinate points The distance is expressed as:

[0066]

[0067] in, It is expressed as the three-dimensional coordinates of the i-th point in the intermediate coordinate system under the laser scanning method, It is expressed as the three-dimensional coordinates of the ath point in the intermediate coordinate system under the photogrammetry method. Let A be the distance threshold. When L≤A, point a and point i can be considered as matching points, and the distance between the matching points is recorded as L. i .

[0068] 1.2. Calculate the overall error value of the point cloud data set, which can be expressed as:

[0069]

[0070] Among them, N p Represents the total number of matched points.

[0071] 2. Calculation based on optimization theory and The gradient descent direction is used as the optimization iteration direction, and the values ​​of R and t are continuously iterated to make f(R, t) as small as possible. When Δf(R, t)≤1×10 -5 The iteration is considered to be terminated when . The coordinate transformation R and t from photogrammetry to laser scanning are recorded as

[0072] 3. Verify whether the R and t obtained above are optimal values.

[0073] Taking the initial coordinate system of the laser scanning method as the fixed coordinate system, the coordinates of all data points in the initial coordinate system of the photogrammetry scanning method are transformed to obtain the position coordinate parameters of the corresponding data points in the fixed coordinate system. The results are extracted and the least squares value of the Euclidean distance of each data point is used as the result of the coordinate matching algorithm, which is then visualized.

[0074] Multiply the initial point cloud coordinate data obtained by photogrammetry by H2 to obtain the coordinate parameters of each point in the intermediate coordinate system of the photogrammetry equipment; multiply Get the position coordinate parameters of the photogrammetric point in the intermediate coordinate system of the laser scanning device; multiply the result by Get the coordinate parameters of the photogrammetric point in the initial coordinate system of the laser scanning device. Specifically, it can be expressed as:

[0075]

[0076] in, Represents the coordinates of the original coordinate data points of photogrammetry in the initial coordinate system of laser scanning; It is the coordinate system registration transformation matrix from photogrammetry to laser scanning. P2 is the coordinate of the original coordinate data point of the laser scanning in the initial coordinate system of the laser scanning.

[0077] According to the distance formula of three-dimensional points, the distance between two points is calculated. When , the distance between the two points can be considered to be close to zero, and the two points can be merged into the same point to eliminate redundant data.

[0078] Calculate the overall error value of the matching points under the unified coordinates, and use this value as the evaluation benchmark for the registration and fusion effect of the two coordinate systems.

[0079]

[0080] 4. Establish a visual expression window to visualize the key point coordinates of the workpiece surface obtained by the converted photogrammetry and the point cloud coordinates obtained by laser scanning in the three-dimensional Cartesian coordinate system, thus realizing the registration and fusion of the two measurement coordinate systems.

[0081] Another embodiment of the present invention further provides a laser vision fusion measurement system, comprising: a laser scanning system, a photogrammetry system, and a processor.

[0082] The laser scanning system is used to perform laser scanning measurement on the surface of the measured object to obtain three-dimensional point cloud data in the laser scanner coordinate system;

[0083] The photogrammetry system is used to perform multi-angle photogrammetry of target points on the surface of the measured object to obtain the three-dimensional coordinates of each target point in the photogrammetry coordinate system;

[0084] Among other things, the processor is used to:

[0085] Perform filtering and denoising on the 3D point cloud data to establish a point cloud database information reflecting the shape of the measured object;

[0086] Establishing target point database information on the surface of the measurement object;

[0087] For the morphological point cloud database information and the target point database information, the common target point in the two measurements is used as the key point of coordinate transformation, an intermediate coordinate system is established, and the three-dimensional coordinate information in the two database information under the intermediate coordinate system is obtained.

[0088] Furthermore, in order to improve the accuracy of fusion measurement, the processor is also used to: align and match the three-dimensional coordinate information of the common target points in the two databases in the intermediate coordinate system, obtain the optimal coordinate transformation matrix between the coordinate systems corresponding to the two database information according to the optimization iteration theory, align the two coordinate systems, and use the laser scanning coordinate system as a reference to realize the fusion measurement of laser scanning and vision.

[0089] The above only expresses the implementation mode of the present invention, and its description is relatively specific and detailed, but it cannot be understood as limiting the scope of the patent. It should be pointed out that for ordinary technicians in this field, several modifications and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention. In addition, the parts not detailed in the present invention are all conventional technologies.

Claims

1. A laser vision fusion measurement method, characterized in that: The method comprises: Step 1: Use a laser scanning system to perform laser scanning measurement on the surface of the measurement object to obtain three-dimensional point cloud data in the laser scanner coordinate system, wherein the three-dimensional point cloud data includes point cloud information on the surface of the measurement object and three-dimensional coordinates of target points, and a plurality of target points are arranged on the surface of the measurement object; Step 2: Filter and denoise the three-dimensional point cloud data to establish a point cloud database information reflecting the shape of the measured object, wherein the point cloud database information includes the denoised surface point cloud information of the measured object and the three-dimensional coordinates of the target point; Step 3: Use a photogrammetry system to perform multi-angle photogrammetry on the target points on the surface of the measurement object, obtain the three-dimensional coordinates of each target point in the photogrammetry coordinate system, and establish a database information of the target points on the surface of the measurement object; Step 4: For the morphology point cloud database information and the target point database information, use the common target point in the two measurements as the key point of coordinate transformation, establish an intermediate coordinate system, obtain the three-dimensional coordinate information in the two database information under the intermediate coordinate system, and realize the fusion measurement of laser scanning and vision.

2. The method according to claim 1, characterized in that Before using the laser scanning system to perform laser scanning measurement on the surface of the measurement object, the method further comprises: pasting a plurality of target points on the surface of the measurement object.

3. The method according to claim 2, characterized in that The spacing between the target points attached to the surface of the measurement object is determined according to the measurement range of the laser scanning system, so that there are always more than three target points in the measurement field when the laser scanning system is measuring.

4. The method according to claim 1, characterized in that: During the laser scanning measurement in step 1, the distance and angle between the laser scanning system and the measured object are adjusted according to the size of the measured object, so that there are more than three target points in the measurement field at all times during the measurement.

5. The method according to claim 1, characterized in that The filtering and denoising process in step 2 uses a radius filtering algorithm to remove outliers and noise points in the scanning results and obtain the morphological point cloud database information of the measured object.

6. The method according to claim 1, characterized in that In step 3, during the photogrammetry, the camera rotates around the object to be measured in sequence for one circle to take multi-angle shots, and each shot contains more than three target points.

7. The method according to claim 1, characterized in that Step 4 specifically includes: After determining the common target points in the two measurement results, three target points on the same surface are randomly selected as the construction elements of the coordinate system, and the intermediate coordinate system is established in the laser scanning coordinate system and the photogrammetry coordinate system respectively; According to the coordinate transformation method, the three-dimensional position coordinates of all points in the morphology point cloud database information are transformed in the same scanning result, the laser scanning coordinate system and the photogrammetry coordinate system are transformed into the intermediate coordinate system, and the three-dimensional coordinate information in the two databases under the intermediate coordinate system is obtained.

8. The method according to any one of claims 1 to 7, characterized in that: After step 4, the method further comprises: Step five: align and match the three-dimensional coordinate information of the common target points in the two databases under the intermediate coordinate system. According to the optimization iteration theory, the optimal coordinate transformation matrix between the coordinate systems corresponding to the two database information is obtained, and the two coordinate systems are aligned and unified. The laser scanning coordinate system is used as the reference to realize the fusion measurement of laser scanning and vision.

9. A laser vision fusion measurement system, characterized in that: The laser vision fusion measurement method according to any one of claims 1 to 8, wherein the system comprises: a laser scanning system, a photogrammetry system, and a processor. The laser scanning system is used to perform laser scanning measurement on the surface of the measured object to obtain three-dimensional point cloud data in the laser scanner coordinate system; The photogrammetry system is used to perform multi-angle photogrammetry of target points on the surface of the measured object to obtain the three-dimensional coordinates of each target point in the photogrammetry coordinate system; Among other things, the processor is used to: Perform filtering and denoising on the 3D point cloud data to establish a point cloud database information reflecting the shape of the measured object; Establishing target point database information on the surface of the measurement object; For the morphological point cloud database information and the target point database information, the common target point in the two measurements is used as the key point of coordinate transformation, an intermediate coordinate system is established, and the three-dimensional coordinate information in the two database information under the intermediate coordinate system is obtained.

10. The system according to claim 9, characterized in that The processor is also used to align and match the three-dimensional coordinate information of the common target points in the two databases under the intermediate coordinate system, obtain the optimal coordinate transformation matrix between the coordinate systems corresponding to the two database information based on the optimization iteration theory, align the two coordinate systems, and use the laser scanning coordinate system as a reference to achieve the fusion measurement of laser scanning and vision.

Images