A skin joint measurement method and device based on data fusion

Abstract

The application provides a skin joint measurement method and device based on data fusion, and belongs to the field of precision measurement. Firstly, the cross section processing is performed on the point cloud of the surface structure light vision measurement, the point cloud obtained by two kinds of measurement methods is identified as the joint feature, and the joint measurement data is obtained by substituting into the skin joint feature mathematical model. Then, the data space alignment is realized by one-to-one corresponding two kinds of structured light measurement results, and the abnormal values of the aligned data are removed. Finally, the processed data is input into the adaptive weighted fusion part, the variance of the measured data of the two is calculated to determine the weighted value, and the data fusion is realized. The application is a measurement method combining the line structured light vision measurement technology and the surface structured light vision measurement technology. The data fusion technology is used to fuse the two kinds of measurement results, the joint feature measurement result is corrected, the limitations of the two kinds of technologies are broken through, the overall measurement precision and measurement efficiency are improved, and a new idea is provided for the skin joint measurement.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of precision measurement, and particularly relates to a skin joint measurement method and device based on data fusion. BACKGROUND

[0002] With the continuous iteration of the performance of aircraft in China, new requirements for digital manufacturing, precise forming and accurate assembly are gradually put forward for the manufacturing and detection of aircraft. Accurate assembly is one of the indispensable steps in the production and manufacturing process of aircraft, and accounts for 40% to 70% of the total workload of aircraft manufacturing. The assembly quality is closely related to factors such as tooling design, assembly sequence, positioning scheme and shape measurement, among which the result of shape measurement is an important indicator for measuring the assembly quality. In the shape measurement work, a large amount of skin joint structure measurement is involved, and the measurement accuracy of the structure is relatively high.

[0003] The measurement methods of skin joint gap and step difference can be summarized into two categories, namely contact measurement and non-contact measurement. The traditional skin joint feature detection method belongs to contact measurement, which mainly relies on detection plug gauges, special gauges and manual observation. This detection method is based on analog measurement results, and it is difficult to accurately describe the joint structure state, and it is difficult to meet the actual needs of manufacturing advanced aircraft in terms of information dimension, detection accuracy and efficiency. It is time-consuming and laborious to measure long skin joints, and the gap and step difference of some complex structures cannot be measured due to the difficulty of inserting the plug gauge. Moreover, this measurement method can only be used for static measurement, and cannot realize real-time online measurement, nor can it quickly import measurement data into a computer for online analysis.

[0004] With the development of digital measurement technology, many non-contact measurement methods have emerged, which greatly improve the assembly quality of skin joint structure, shorten the aircraft assembly cycle and reduce the difficulty of shape measurement work. Non-contact measurement methods are generally divided into passive visual measurement and active visual measurement. Passive visual measurement does not require special lighting devices such as laser emitters and projectors, and only uses a camera to capture the image of the object to be measured. The pose relationship between the object to be measured and the camera is established, and the three-dimensional information of the object to be measured can be obtained. However, it is greatly affected by environmental light, and it is difficult to achieve high-quality matching for objects with small features, which reduces the accuracy of subsequent calculation results. As a representative of active visual measurement methods, structured light visual measurement technology is widely used. This technology is guided by the principle of triangular optics, and through camera calibration and structured light plane calibration, two-dimensional pixel coordinates are converted into corresponding three-dimensional coordinates. The measurement result has high resolution, and the measurement process is fast and not affected by external temperature, light and other environmental factors. Therefore, it is feasible to use structured light visual measurement technology for skin joint detection.

[0005] Beijing University of Aeronautics and Astronautics, Nanjing University of Aeronautics and Astronautics, Chinese Academy of Sciences, and various application and research units of aviation industry have also carried out related researches on skin joint detection. Xu Dasha of Beijing University of Aeronautics and Astronautics, Zhang Ka of Nanjing University of Aeronautics and Astronautics, and Wu Xingjiang of aviation industry have all used line structured light vision measurement technology to detect the skin joint, but this technology needs to position the feature area, requires that the laser beam projected by the line structured light laser is perpendicular to the surface of the skin joint, is easily affected by the angle between the joint and the laser beam, is difficult, and has low efficiency due to the measurement of each scanning position. In the aspect of area structured light vision measurement, Xia Renbo and Chen Songlin of Chinese Academy of Sciences use the stripe projection technology to measure the skin joint features, locate the joint position according to the corresponding relationship between the three-dimensional point cloud data and the image, determine the edge points on both sides of the joint, calculate the equivalent of the joint features, and improve the stripe projection measurement method, thereby reducing the phase error and improving the adaptability of the method to the surface of the measured object. Compared with the line structured light vision measurement, the three-dimensional data obtained by this technology is more dense, the joint size in the measured field of view only needs to be measured once, and the working efficiency is greatly improved, but the measurement accuracy of smaller gaps and steps is low. In summary, the joint detection system using line structured light sensor alone needs to be improved in terms of detection efficiency, which is not conducive to the overall digitization of skin joint detection. The joint detection system using area structured light sensor alone can improve the detection efficiency to a certain extent, but has defects in terms of measurement accuracy. SUMMARY

[0006] In order to overcome the shortcomings of the prior art, the present application provides a skin joint measurement method based on data fusion.

[0007] In order to overcome the shortcomings of the prior art, the present application provides a skin joint measurement method based on data fusion.

[0008] In order to achieve the above purpose, the present application provides the following technical scheme:

[0009] A skin joint measurement method based on data fusion, comprising the following steps:

[0010] Obtaining line structured light vision measurement point cloud data and area structured light vision measurement point cloud data;

[0011] extracting line joint features in the line structured light vision measurement point cloud data;

[0012] splitting discrete point clouds in the area structured light vision measurement point cloud data into line point clouds, and extracting edge points of the line point clouds to obtain area joint features;

[0013] substituting the line joint features and the area joint features into a skin joint feature mathematical model respectively to obtain line joint light measurement data and area joint light measurement data;

[0014] aligning the line joint light measurement data and the area joint light measurement data;

[0015] using an adaptive weighted fusion algorithm to fuse the aligned joint measurement data to obtain joint step difference values and gap values.

[0016] Preferably, the skin joint feature mathematical model is:

[0017]

[0018] wherein p is a joint point cloud density, 1≤i≤n, n is a number of joint edge points, gap i is a distance from each edge point to a corresponding fitting straight line.

[0019] Preferably, line structured light vision measurement images and area structured light vision measurement images are collected by using a line structured light vision measurement sensor and an area structured light vision measurement sensor respectively, and the line structured light vision measurement point cloud data and the area structured light vision measurement point cloud data are obtained by performing coordinate conversion processing on the line structured light vision measurement images and the area structured light vision measurement images.

[0020] Preferably, the laser stripes of the line structured light vision measurement point cloud data have four states in a pixel coordinate system, which are: having a step difference and having a gap zero point, having a step difference and not having a gap zero point, not having a step difference and having a gap zero point, and not having a step difference and not having a gap zero point.

[0021] Preferably, a joint feature recognition algorithm based on pixel deviation anomaly is used to extract the line joint features in the line structured light vision measurement point cloud data, and the method comprises the following steps:

[0022] For the phenomenon of having a step difference and having a gap zero point, an improved slope algorithm is used to determine the critical points on both sides according to the position of the calculated slope change;

[0023]

[0024] In the formula, k(i) is an average slope, i(i≥2) is a pixel point row coordinate, x(i-1), x(i+1), x(i-2), and x(i+2) are column coordinates of the i-1, i+1, i-2, and i+2 corresponding pixel points, respectively;

[0025] For the phenomenon of step difference and no gap zero point and no step difference and no gap zero point, the critical point is determined by searching for the broken missing of the pixel column coordinates, the Euclidean distance of the adjacent two points is calculated, and the edge points on both sides are determined;

[0026] For the phenomenon of no step difference and gap zero point, first, the no step difference and gap zero point is converted into no step difference and no gap zero point, the critical point is determined by searching for the broken missing of the pixel column coordinates, the Euclidean distance of the adjacent two points is calculated, and the edge points on both sides are determined.

[0027] Preferably, the point cloud cross-section slicing method is used to split the discrete point cloud in the surface structure light vision measurement point cloud data into a plurality of linear point clouds, and the edge points of the plurality of linear point clouds are extracted by using the joint feature recognition algorithm based on angle mutation to obtain the surface joint features;

[0028] The acquisition process of the plurality of linear point clouds is as follows:

[0029] The surface structure light vision measurement point cloud data is processed to obtain a height information topography of the data;

[0030] In the height information topography, the y coordinate in the surface structure light vision measurement point cloud data is taken as a feature for segmentation, and the X-Z plane is taken as a cross section for slicing processing of the point cloud data to obtain linear point clouds;

[0031] The y pixel coordinate in the height information topography data is taken as a feature label, and the data containing the same y coordinate is classified into a class, thereby obtaining a plurality of linear height information topographies;

[0032] The joint camera calibration parameters of each group of data are obtained to obtain a plurality of linear point clouds.

[0033] Preferably, the alignment processing of the linear joint seam light measurement data and the surface joint seam light measurement data specifically includes the following steps:

[0034] The laser stripe center pixel coordinates of the first laser stripe graph of the linear structure light measurement data are acquired;

[0035] The acquired laser stripe center pixel coordinates are aligned with the height information matrix in the surface structure light measurement data;

[0036] The point cloud cross-section slicing processing algorithm is used to acquire the point cloud coordinates in the surface structure light seam point cloud corresponding to the first seam data measured by the linear structure light;

[0037] Add one to the pixel coordinate, repeat the second and third steps until the two kinds of structured light measurement data correspond completely.

[0038] Preferably, the aligned joint measurement data is cleaned before fusion, and the outliers are removed, and the specific steps are as follows:

[0039] Sort the n times of data collected by the line structured light vision measurement sensor and the area structured light vision measurement sensor from small to large as follows: i1 ,x i2 ,......,x in , where x in is the maximum value, and x i1 is the minimum value.

[0040] Calculate the average value of the sequence and the standard deviation s i .

[0041] Calculate the statistics G max and G min of the maximum and minimum values.

[0042]

[0043] Set G(n,a) as the critical value, where a is the significant level, and the specific G(n,a) can be obtained by determining the corresponding n and a and consulting the Grubbs criterion table; when G max ≥G min and G max >G(n,a), x in is considered as an outlier; when G min ≥G max and G min >G(n,a), x i1 is considered as an outlier; and then the outliers are removed.

[0044] Then, the second and third steps are repeated on the remaining data until no outliers appear, and the removal of outliers is completed.

[0045] Preferably, the aligned two kinds of joint measurement data are fused by using a distributed two-level data fusion algorithm to obtain the fused joint step difference value and gap value, which specifically includes the following steps:

[0046] Solve the direction variance of the line structured light vision measurement sensor and the area structured light vision measurement sensor

[0047] Calculate the weighted coefficient of the direction of the line structured light vision measurement sensor and the area structured light vision measurement sensor.

[0048]

[0049] wherein, p = 1 or 2,

[0050] The cleaned line joint light measurement data and the surface joint light measurement data are substituted into the fusion value calculation formula to obtain a fusion result;

[0051]

[0052] In the formula, X r is a weighted fusion value, W p is a corresponding weighting coefficient, X p is a corresponding measurement value; p = 1 or 2; wherein, X1 and X2 are the cleaned line joint light measurement data and the surface joint light measurement data, and W1 and W2 are weighting factors corresponding thereto.

[0053] Another object of the present application is to provide a skin joint measurement device based on data fusion, comprising:

[0054] A line structured light vision measurement sensor and a surface structured light vision measurement sensor respectively collect a line structured light vision measurement image and a surface structured light vision measurement image;

[0055] A data processing module is configured to perform coordinate conversion processing on the line structured light vision measurement image and the surface structured light vision measurement image to obtain line structured light vision measurement point cloud data and surface structured light vision measurement point cloud data;

[0056] A feature extraction module is configured to perform feature recognition processing on the line structured light vision measurement point cloud data and perform cross-section processing on the surface structured light vision measurement point cloud data and then perform feature recognition processing to extract line joint features and surface joint features;

[0057] A data calculation module is configured to substitute the line joint features and the surface joint features into a skin joint feature mathematical model to obtain line joint light measurement data and surface joint light measurement data;

[0058] A data alignment module is configured to perform alignment processing on the line joint light measurement data and the surface joint light measurement data;

[0059] A data fusion module is configured to fuse the two kinds of joint measurement data after alignment by using an adaptive weighted fusion algorithm to obtain a fusion value.

[0060] The skin joint measurement method and device based on data fusion provided by the present application have the following beneficial effects:

[0061] (1) The measurement system can be arranged at a position suitable for three-dimensional reconstruction, which is conducive to measuring the skin joint.

[0062] (2) The skin joint feature mathematical model established by the application simplifies the acquisition of joint measurement data.

[0063] (3) The application divides the surface structure light into a plurality of linear point clouds by the height information model of the surface structure light, which is beneficial to the extraction of joint features and the later alignment and fusion.

[0064] (4) The application adopts distributed secondary data fusion, combines data alignment, data cleaning and data fusion, and provides a strong guarantee for fusion accuracy.

[0065] (5) The application combines the advantages of line structure light vision measurement joint and surface structure light vision measurement joint, balances the measurement results, greatly improves the overall measurement accuracy and measurement efficiency, and has a good pioneer significance for developing digital and intelligent skin joint detection instruments. BRIEF DESCRIPTION OF DRAWINGS

[0066] In order to more clearly illustrate the embodiments of the application and the design scheme, the drawings required by the embodiments will be briefly introduced below. The drawings in the following description are only part of the embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0067] Figure 1 The flowchart of the skin joint measurement method based on data fusion provided by the application is shown in the figure.

[0068] Figure 2 The schematic diagram of the skin joint feature mathematical model of the application is shown in the figure.

[0069] Figure 3 The edge point projection diagram of the application is shown in the figure.

[0070] Figure 4 The laser stripe type diagram of the application is shown in the figure; wherein, (a) has a step difference and a gap zero point, (b) has a step difference and no gap zero point, (c) has no step difference and a gap zero point, and (d) has no step difference and no gap zero point.

[0071] Figure 5 The flowchart of the line structure light joint feature extraction algorithm of the application is shown in the figure.

[0072] Figure 6 The joint information diagram of the application is shown in the figure; wherein, (a) is a height information topographic map, and (b) is a joint three-dimensional point cloud diagram.

[0073] Figure 7 The y coordinate data comparison diagram of the application is shown in the figure; wherein, figure (a) is a y coordinate data diagram in a three-dimensional point cloud, and figure (b) is a y coordinate data diagram in a height topography.

[0074] Figure 8Line point cloud diagram for the present invention;

[0075] Figure 9 Left side point cloud data diagram for the present invention;

[0076] Figure 10 Data alignment flow chart for the present invention;

[0077] Figure 11 Unaligned point cloud diagram for the present invention;

[0078] Figure 12 Aligned point cloud diagram for the present invention;

[0079] Figure 13 Measurement system structure schematic diagram for the present invention;

[0080] Figure 14 Physical diagram of a joint standard part to be detected for embodiment 1 of the present invention;

[0081] Figure 15 Point cloud alignment diagram for embodiment 1 of the present invention; wherein (a) joint #1 alignment diagram, (b) joint #2 alignment diagram, (c) joint #3 alignment diagram, (d) joint #4 alignment diagram;

[0082] Figure 16 Gap value fusion result diagram for embodiment 1 of the present invention; wherein (a) joint #1 data diagram, (b) joint #2 data diagram, (c) joint #3 data diagram, (d) joint #4 data diagram;

[0083] Figure 17 Step difference value fusion result diagram for embodiment 1 of the present invention; wherein (a) joint #1 data diagram, (b) joint #2 data diagram, (c) joint #3 data diagram, (d) joint #4 data diagram;

[0084] Figure 18 Physical diagram of a skin to be detected for embodiment 2 of the present invention;

[0085] Figure 19 Face structure light measurement method diagram for embodiment 2 of the present invention;

[0086] Figure 20 Line structure light measurement method diagram for embodiment 2 of the present invention;

[0087] Figure 21 Face structure light measurement value for embodiment 2 of the present invention; wherein (a) face structure light gap value, (b) face structure light step difference value;

[0088] Figure 22 Point cloud alignment diagram for embodiment 2 of the present invention;

[0089] Figure 23This is a fusion result diagram of Example 2 of the present invention; wherein (a) is position #1, (b) is position #2, and (c) is position #3;

[0090] Figure 24 Graphs showing rough measurement results of Example 2 of the present invention; (a) a graph showing a change in gap value, and (b) a graph showing a change in step difference value. DETAILED DESCRIPTION

[0091] In order to enable those skilled in the art to better understand the technical solution of the present invention and to be able to implement it, the present invention is described in detail below with reference to the accompanying drawings and specific embodiments. The following embodiments are only used to more clearly illustrate the technical solution of the present invention and are not intended to limit the scope of protection of the present invention.

[0092] Example 1

[0093] The present invention provides a skin seam measurement method based on data fusion, specifically Figure 1 As shown, the following steps are included:

[0094] Step 1: Acquire seam measurement data. The specific steps are as follows:

[0095] Step 1.1: Collect line structured light vision measurement point cloud data and surface structured light vision measurement point cloud data. In reverse engineering, the point data set of the product surface obtained by measuring instruments is also called a point cloud.

[0096] Step 2: Perform feature recognition processing on the line structured light vision measurement point cloud data, perform cross-section processing on the surface structured light vision measurement point cloud data, and then perform feature recognition processing to extract seam features. Specifically:

[0097] (1) In the online structured light vision measurement part, the seam features are extracted using a seam feature recognition algorithm based on pixel deviation anomaly.

[0098] The four states of the laser stripes in the pixel coordinate system are processed separately. Figure 4 As shown. Figure 4 In the phenomenon shown in (a), the seam exhibits a cliff-like slope change. By calculating the slope point by point, a sharp abrupt change in slope, one positive and one negative, should be observed. This point is the critical point. However, the traditional slope formula only uses two points to calculate the slope, which is susceptible to random errors and unsuitable for determining the slope of laser streak curves. To avoid local anomalies caused by fluctuating noise and improve the accuracy of critical point extraction, this embodiment employs an improved slope algorithm. This algorithm determines the critical points on both sides based on the calculated slope change location.

[0099]

[0100] In the formula, k(i) is an average slope, i(i >= 2) is a pixel point row coordinate, x(i-1), x(i+1), x(i-2) and x(i+2) are column coordinates of i-1, i+1, i-2 and i+2 corresponding pixel points respectively.

[0101] For Figure 4 (b) and (d), the critical point can be determined by searching for the broken missing of the pixel column coordinates, from Figure 4 (b) and (d), the Euclidean distance of the edge points on both sides of the broken point is much larger than that of other adjacent two points, so the Euclidean distance of the adjacent two points can be calculated to determine the two side edge points.

[0102] For Figure 4 (c), the present application converts it into Figure 4 (d) and then extracts the features.

[0103] The flow chart of the seam feature extraction algorithm of the line structured light is as shown in Figure 5 In the step of extracting the critical point and the edge point, according to the experience, the distance between the edge point and the critical point is usually twice the point cloud density, so the position of one point can be determined at the same time as the position of the other point is determined. Then, the point cloud data is divided into three parts with the two critical points as the demarcation points, and the gap zero point and the two side points are separated, and the line fitting is performed on the point cloud data on both sides to obtain the required line equation. Finally, the corresponding step difference value and gap value are obtained by substituting the features into the mathematical model of the skin seam feature.

[0104] (2) In the plane structured light vision measurement part, the seam feature points are extracted by using the point cloud cross-section slicing processing algorithm and the seam feature recognition algorithm based on angle mutation.

[0105] The present application uses the point cloud cross-section slicing method to split the discrete point cloud of the plane structured light into a plurality of linear point clouds based on the reference line structured light technology measurement idea, so as to reduce the calculation amount and improve the accuracy. According to Figure 6 It can be seen that, under ideal conditions, the y coordinate in the plane structured light vision measurement point cloud data is used as a feature for segmentation, and the X-Z plane is used as a cross section, so that the point cloud data can be sliced to obtain linear point clouds. Figure 6 In the "height information topography" in (a), the x and y coordinates are pixels, that is, pixels. Figure 6 In (b), the x, y and z coordinates are all in mm. As shown in Figure 7 It can be seen that, under ideal conditions, the y coordinate in the plane structured light vision measurement point cloud data is used as a feature for segmentation, and the X-Z plane is used as a cross section, so that the point cloud data can be sliced to obtain linear point clouds.

[0106] Therefore, the y pixel coordinates in the high profile information map are taken as a characteristic label, data with the same y are classified into one category, thereby obtaining a plurality of line-shaped height information profile maps, and a plurality of line-shaped point clouds can be obtained by combining the data and the camera calibration parameters. Figure 8 As shown in the figure, this is a single line-shaped point cloud map after processing.

[0107] The application extracts the reference line structure light joint by extracting the edge points of the line-shaped point cloud after the point cloud section slicing processing in the manner of angle mutation between point clouds, thereby determining the joint area.

[0108] As shown in the figure, point D in the figure is the joint critical point, and point E in the figure is the joint edge point. The specific judgment algorithm is as follows: Figure 9

[0109] ①Taking the first three points A, B and C in the figure as an example, the distance d between point C and the straight line AB is calculated, and the sine value of the supplementary angle of ∠ABC is obtained by comparing the modulus of the line segment BC, and the angle of the supplementary angle of ∠ABC is obtained by the inverse trigonometric function.

[0110] ②Then, the threshold θ is set as a judgment condition, if the angle is less than θ, the points A, B and C are merged to fit a straight line, and the next round of adjacent line segment angle is calculated; if the angle is greater than θ, it is considered that the joint edge point is found. By observing the joint point cloud data, the threshold θ is set to 20°.

[0111] ③Repeat the above steps until the left edge point of the joint is found, then stop the cycle.

[0112] ④The right side data of the joint can be obtained by repeating the above steps from the right side, and the coordinates of the critical point can be calculated according to the distance relationship between the edge point and the critical point.

[0113] ⑤Finally, the fitting straight line equation, critical point coordinates and edge point coordinates are substituted into the joint skin joint feature mathematical model, and the step difference value and gap value can be calculated.

[0114] Step 1.3, substitute the joint feature into the skin joint feature mathematical model to obtain the specific measurement value.

[0115] As shown in the figure, the process of establishing the skin joint feature mathematical model is as follows: Figure 2

[0116] Wherein, the circle point indicates the measured joint point cloud, the solid line indicates the real joint; P a0 , P a1 is the curvature mutation point of the measured joint point cloud, that is, the joint critical point; P b0 , P b1 ​​is the edge contour point of the measured seam point cloud, i.e., the seam edge point; L1 and L2 are the straight lines fitted by the points on both sides that are approximately on the same straight line.

[0117] The present invention is based on d a0 with d a1 Half of the sum is taken as the measured standard order difference. Figure 2 The midpoint only has two-dimensional information, so let point P a0 The coordinates of (x a0 ,y a0 ), click P a1 The coordinates of (x a1 ,y a1 ), the straight line L1 is A1x+B1y+C1=0, and the straight line L2 is A2x+B2y+C2=0, then the measured standard order difference value can be expressed as:

[0118]

[0119] To calculate the measured standard gap value, Figure 2 Lieutenant point P b0 Projecting onto line L2 yields point P b0 ', click P b1 Projecting onto line L1 yields point P b1 ',like Figure 3 As shown. The calculation of the measured standard order difference is the same as that of the measured standard order difference. b0 To the fitting straight line P b1 P b1 'Distance or point P b1 To the fitting straight line P b0 P b0 The distances are all measured standard gap values. Since there are many edge points, all the distances are accumulated and summed, and the average is used as the measured standard gap, which can be expressed as follows:

[0120]

[0121] In formula (3), 1≤i≤n, n is the number of seam edge points, gap i is the distance from each edge point to the corresponding fitting line.

[0122] like Figure 3 As shown in the figure, the gap in the actual seam point cloud inevitably affects the edge point extraction, making the measured value larger than the true value. The measurement error is the seam point cloud density ρ, so the measured standard gap is expressed as:

[0123]

[0124] After differential processing of the point cloud data, the point cloud density is approximately 0.08 mm.

[0125] In summary, the joint measurement problem is converted into obtaining critical point, edge point coordinates and fitting straight line equation according to the above model, and finally the extracted line joint characteristics and surface joint characteristics are substituted into formula (4) to solve.

[0126] Step 2, align the line joint light measurement data and the surface joint light measurement data, the alignment method of the present application is to combine the first laser stripe center pixel coordinates collected by the line structured light with the height information model of the surface structured light, to determine the linear point cloud corresponding to the surface structured light data of the first line structured light data, and to align the remaining data by accumulating the coordinates of the pixels, the specific steps are as follows:

[0127] (1) Obtain the laser stripe center pixel coordinates of the first laser stripe image of the line structured light measurement data.

[0128] (2) Align the obtained laser stripe center pixel coordinates with the height information matrix in the surface structured light measurement data.

[0129] (3) Use the point cloud cross-section slicing processing algorithm to obtain the point cloud coordinates in the surface structured light joint point cloud corresponding to the first joint data measured by the line structured light.

[0130] (4) Add one to the pixel coordinates, repeat steps (2) and (3) until the two kinds of structured light measurement data correspond completely, the specific process is shown in Figure 10 .

[0131] The specific processing process of the point cloud cross-section slicing processing algorithm is as follows:

[0132] Before data fusion of the line and surface two kinds of structured light sensor detection data, due to different poses, the measured data coordinate system is also different, therefore, in order to ensure the consistency and effectiveness of the measured data of the two kinds of sensors, the two kinds of data need to be aligned, and the measurement unit needs to be consistent. Generally, data alignment is divided into time and space alignment methods, based on the actual detection situation, the present application adopts the space alignment method to correspond the two kinds of structured light measurement data.

[0133] As shown in Figure 11 , this is the point cloud graph before the alignment of the two kinds of structured light measurement standard part data. In order to reduce the calculation amount, the point cloud data shown in the figure is cropped, the area of the surface structured light measurement data point cloud is larger, and the area of the line structured light measurement data point cloud is smaller. The line structured light vision measurement sensor adjusts the high-precision moving platform to ensure that the distance between lines is equal to the point cloud density, when the single line structured light measurement data is spliced with a step of 0.08mm, the Figure 11The line structured light summary point cloud consistent with the point cloud density of the surface structured light measured data.

[0134] According to the above situation, the alignment of the line and surface structured light module measured data can be converted into the alignment of the first joint point cloud of the two structured light module measured data. After determining the first data of the two structured light, the corresponding coordinates of the surface structured light point cloud of the other line structured light point cloud can be obtained by accumulating the coordinates according to the consistent interval principle. The data alignment algorithm flowchart is shown as follows:

[0135] As shown in Figure 10 , in the actual measurement process, the height information matrix generated by the surface structured light vision measurement sensor is the pixel size of the picture, which reflects the height information of each pixel point in the picture. Therefore, in the ideal case, only the pixel position of the first joint data measured by the line structured light vision measurement sensor needs to be found, and the average row pixel coordinates of the laser stripe center in the first joint data measured by the line structured light are input into the surface structured light point cloud slice processing to determine the position together with the height information matrix, so as to obtain the corresponding joint point cloud of the line structured light joint point cloud in the surface structured light measured data. Then, the corresponding positions of other data are obtained by accumulating (determined by the moving direction of the line structured light vision measurement sensor), until all the structured light measurement data are corresponding, which indicates that the data alignment is completed.

[0136] Finally, taking the coordinate system of the surface structured light as the reference, the corresponding coordinate relationship between the lines measured by the line and surface structured light is calculated by using the line structured light measurement data, and the coordinate conversion formula is as follows:

[0137] P o =P s *R+T (5)

[0138] In formula (5), P o is the target matrix; P s is the original matrix; R matrix and T matrix are respectively the rotation matrix and the translation matrix, also known as the coordinate correspondence relationship. In order to ensure the consistency of the point cloud, the feature vectors composed of the edge points on both sides of each group are used as the target vectors, and the edge point feature vectors measured by the surface structured light are used as the original vectors. The corresponding R and T matrices are calculated by substituting the above formula. Since the laser lines are parallel to each other and have equal intervals in the line structured light measurement, the corresponding R matrix of the line structured light measured data when converted to the coordinate system of the surface structured light is the same, and the T matrix changes according to the interval size.

[0139] According to the R and T matrices calculated in the above steps, the data of each point measured by the line structured light are substituted into formula (5) for coordinate transformation. After transformation and alignment, the three-dimensional point cloud of line and surface structured light is obtained as follows: Figure 12 shown.

[0140] Step 3: Clean the seam measurement data and remove abnormal values. The specific steps are as follows:

[0141] (1) Sort the n data collected by the two structured light vision measurement technologies into x i1 ,x i2 ,......,x in , where x in is the maximum value, x i1 is the minimum value;

[0142] (2) Calculation sequence Average value and s i Standard deviation;

[0143] (3) Calculate the statistics G of the maximum and minimum values max With G min ;

[0144]

[0145] (4) Set G(n,a) as the critical value, where a is the significance level. The specific G(n,a) can be obtained by determining the corresponding n and a and consulting the Grubbs criterion table. max ≥G min And G max >G(n,a), then x in is an outlier; when G min ≥G max And G min >G(n,a), then x i1 The outliers are then removed, and steps (2) and (3) are repeated for the remaining data until no outliers appear. This concludes the outlier removal algorithm.

[0146] Step 4: Fuse the seam measurement data to obtain the fusion value.

[0147] The present invention adopts an adaptive weighted fusion algorithm to fuse line and surface structured light vision measurement data. The specific steps are as follows:

[0148] (1) Solving the directional variance of line structured light vision measurement sensors and surface structured light vision measurement sensors

[0149] (2) Calculate the weighting coefficients of each sensor direction;

[0150]

[0151] wherein p = 1 or 2.

[0152] (3) Substitute the cleaned-up line joint light measurement data and the surface joint light measurement data into formula (8) to obtain the fusion result.

[0153]

[0154] wherein X r is the weighted fusion value, W p is the corresponding weighting coefficient, X p is the corresponding measurement value; p = 1 or 2; wherein X1, X2 are the cleaned-up line joint light measurement data and the surface joint light measurement data, and W1, W2 are the corresponding weighting factors.

[0155] Both the joint step difference value and the joint gap value can be applied to formula (8). For example, when X r is the fusion value of the joint step difference value, X p is the joint step difference value measured by the corresponding sensor, and W p is the weighting coefficient when the joint step difference is measured by the corresponding sensor. Table 6 presents the weighting coefficients in the two cases of the joint step difference value and the gap value.

[0156] Fusion result analysis:

[0157] To analyze the overall detection effect, the system measurement error is introduced as the evaluation standard. When the confidence probability is 95.44%, the 2 times of the standard deviation between the measurement value and the true value is taken as the judgment standard for calculating the system measurement error, and the formula can be expressed as:

[0158]

[0159] wherein ξ is the system measurement error, x i is the i-th group of measurement values, Z is the true value, and n is the number of groups of measurements. The system measurement error value is solved by the above method, and the overall measurement effect of the system can be evaluated.

[0160] The fusion result is as shown in Figure 16 , Figure 17 and Tables 7 and 8 below.

[0161] From Figure 16 , Figure 17It can be seen that, in the gap value fusion part, the fusion value is closer to the surface structure light measurement value; in the step value fusion part, the fusion value is closer to the line structure light measurement value. And compared with the two structure lights before fusion, the measurement value after fusion has smaller fluctuation. The specific fusion result analysis is shown in Tables 7 and 8. It can be obtained from the fusion result table that, compared with the structure light measurement alone, the measurement mean value after fusion is closer to the theoretical value, and the mean error is smaller. And when the two measurement mean values are located on both sides of the true value respectively, the fused result is better, in which the #2 gap, the #3 gap, the #4 gap and the #1 step conform to this rule.

[0162] The maximum measurement error of the fused gap value system is 0.0887 mm, and the maximum measurement error of the fused step value system is 0.0717 mm. In this embodiment, the effective number 0.09 mm is taken as the measurement error of the fused gap value, and the effective number 0.08 mm is taken as the measurement error of the fused step value. From the fusion result, this fusion method balances the two structure light measurement results, improves the overall measurement accuracy, and verifies the effectiveness of the fusion method.

[0163] To solve the problem of the joint feature, the embodiment combines the structure light measurement principle and the data fusion theory, and designs a skin joint measurement device combining line structure light vision measurement and surface structure light vision measurement, so as to realize comprehensive and efficient detection of the joint feature. The device mainly includes a line structure light vision measurement sensor, a surface structure light vision measurement sensor, a motion control system, a data processing module, a feature extraction module, a data calculation module, a data alignment module and a data fusion module.

[0164] Specifically, the line structure light vision measurement sensor and the surface structure light vision measurement sensor respectively collect line structure light vision measurement images and surface structure light vision measurement images; the data processing module is used for coordinate conversion processing of the line structure light vision measurement images and the surface structure light vision measurement images to obtain line structure light vision measurement point cloud data and surface structure light vision measurement point cloud data; the feature extraction module is used for feature recognition processing of the line structure light vision measurement point cloud data, and feature recognition processing after cross-section processing of the surface structure light vision measurement point cloud data, to extract line joint features and surface joint features; the data calculation module is used for substituting the line joint features and the surface joint features into a skin joint feature mathematical model to obtain line joint light measurement data and surface joint light measurement data; the data alignment module is used for alignment processing of the line joint light measurement data and the surface joint light measurement data; and the data fusion module is used for fusion of the two joint measurement data after alignment by using a self-adaptive weighted fusion algorithm to obtain a fusion value.

[0165] This embodiment utilizes the line structured light vision measurement image and the surface structured light vision measurement image captured by the line structured light vision measurement sensor and the surface structured light vision measurement sensor, respectively, and performs coordinate conversion on the line structured light vision measurement image and the surface structured light vision measurement point cloud data to obtain the line structured light vision measurement point cloud data. For line structured light, the pixel coordinates of the laser stripes in the image need to be converted into the X and Y coordinates of the three-dimensional data through an algorithm, and then the corresponding Z coordinate is obtained by calibrating the line structured light vision measurement sensor.

[0166] For surface structured light, it is necessary to extract the phase information of the fringes in the image, obtain the Z information through the surface structured light calibration algorithm, and then use the camera calibration to obtain the X, Y coordinates.

[0167] Specifically, if Figure 13 As shown, the line structured light vision measurement sensor primarily consists of a camera and a line laser, while the area structured light vision measurement sensor primarily consists of a camera and a projector. The motion control system consists of a high-precision guide rail platform and a motor control box. The high-precision guide rail platform is an existing dual-guide rail and lead screw platform. A support frame is fixed to the platform's loading platform. The camera is attached to one side of the support frame, while the projector and line laser are attached to the other side. The projector is positioned above the line laser.

[0168] During the actual detection process, the line structured light vision measurement sensor collects the original image of the laser stripes containing the seam feature information, and the surface structured light vision measurement sensor captures the image of the object to be measured with the grating stripes projected. The collected modulated image is transmitted to the computer, and the image is processed according to the corresponding algorithm to extract the seam feature data. The data measured by each sensor is then input into the data fusion model to achieve accurate detection of the seam features.

[0169] The camera is an important acquisition component in structured light vision measurement, and its quality determines the accuracy of the captured image. For example, the resolution of the camera has the greatest impact on the sharpness of the captured image. In the joint measurement device of the embodiment, when the camera captures the joint laser image or the joint grating projection image, if the camera lens performance is low, it will directly affect the quality of the original image, thereby affecting the joint feature extraction and leading to the influence of the joint equivalent measurement result. To meet the measurement requirements of the joint comprehensive system, the ace U series product acA2440-75um industrial camera of Basler Company in Germany is selected in this embodiment. The resolution of this camera can be adjusted to a maximum of 2448×2048 pixels, and the rate of capturing one frame can reach a maximum of 75 fps, which provides strong support for fast acquisition of joint feature images. In terms of lens, the 8mm focal length lens manufactured by Computar Company is selected, and its model is M0828-MPW3. This lens can adjust the focal length and aperture according to the on-site measurement environment, and is flexible to use.

[0170] The line structured light vision measurement sensor of the embodiment mainly consists of a camera and a line laser. The quality of the laser stripe projected by the line laser is related to the three-dimensional reconstruction effect. When the laser line brightness is uneven, the extracted laser stripe center line deviates from the ideal characteristics, thereby affecting the three-dimensional point cloud data and leading to a decrease in the reconstruction accuracy. To ensure that the camera can capture a laser stripe with uniform brightness and fine line width, and achieve the effect of high-precision measurement of the skin joint, the embodiment selects a type of one-word laser of Powell HW650AB100-16GD-WLD. This laser has the advantages of adjustable line width, small volume, easy installation, and strong stability, which meets the measurement requirements of the embodiment.

[0171] The surface structured light vision measurement sensor of the embodiment mainly consists of a projector and a camera. The role of the projector in this module is to project a surface structured light pattern. There are four kinds of projection principles of the projector. The projector based on the cathode ray tube (CRT) and the silicon-based liquid crystal display (LCOS) principle does not meet the test requirements. The projector based on the liquid crystal display (LCD) and the digital light processing (DLP) principle has relatively high contrast, resolution, and light conversion efficiency, and the DLP projector can be zoomed to stabilize the image quality. In summary, to project a grating stripe pattern with high quality and small distortion, the embodiment selects the DLP Lightcrafter4500 projector based on the TI Texas Instruments development platform as the grating projection device. This projector has a single-channel light source design and zero-offset light path, which can effectively improve the light source utilization efficiency, and is equipped with two I / O triggers, which can realize the linkage with the camera to collect the grating stripe image through programming.

[0172] The embodiment is corresponding to the data measured by each line structured light stripe and the surface structured light grating stripe, and requires strict accuracy of the guide rail moving platform to stabilize the motion track of the line structured light. In order to meet the requirement, the embodiment selects a 23HS3430D8B stepping motor. The laser is fixed on the moving platform, and the double guide rail screw of the moving platform is used for translation. The whole measurement process is free of shaking and noise, the laser line can move stably on the surface of the measured object, and the system error caused by uneven movement of the laser line is reduced.

[0173] The motion control of the motor is also the key to the stable movement of the laser. In the motion control part of the embodiment, the SC300 series controller of Zhuoli Han Guang is selected to drive the stepping motor. The controller can realize point-to-point positioning movement of three axes X, Y and Z in space, and can move a small distance according to the measurement requirement, and has high accuracy.

[0174] The skin joint feature measurement device built by the above equipment is spliced with the two structured light vision measurement sensors on the object table of the guide rail moving platform. The total stroke length of the moving platform is 300 mm, which can meet the detection requirement of the line structured vision measurement sensor. According to the characteristics of the surface structured light vision measurement and the line structured light vision measurement, the motion speed and step length of the moving platform can be controlled to meet the requirement of joint detection.

[0175] Embodiment 1

[0176] In the embodiment, the joint measurement method of the application is used to detect the joint standard part.

[0177] (1) Description of the standard part to be detected:

[0178] The standard part to be detected is shown in FIG. 1, wherein the step difference of joint #1 is 1.5 mm and the gap value is 3 mm; the step difference of joint #2 is 1 mm and the gap value is 2 mm; the step difference of joint #3 is 0.5 mm and the gap value is 1 mm; and the step difference of joint #4 is 0 mm and the gap value is 0.6 mm. Figure 14 (2) Detection method:

[0179] In the application, the moving mode of the moving platform is set, so that the laser stripe interval and the point cloud density collected by the line structured light measurement part are consistent. The motion control box is set to drive the line structured light vision measurement sensor to move at a speed of 0.4 mm / s in the direction shown in the figure, and the camera is set to take a laser stripe image every 200 mm during the movement. The detection method of the surface structured light vision measurement sensor is to set the camera to trigger mode, so that the projector projection grating pattern and the camera shooting are synchronous behaviors.

[0180] (3) Joint measurement data acquisition:

[0181] ​

[0182] ①Line structured light vision measurement data acquisition:

[0183] In the detection process of the butt joint standard specimen by the line structured light vision measurement sensor, 200 groups of laser stripe data were collected, 50 groups for each of the four kinds of joints. Since each joint size in the measured joint standard specimen is uniform, it can be considered that the scanning data is a repeated measurement group of the butt joint standard specimen. The joint feature recognition algorithm based on pixel deviation anomaly is used to extract the joint feature, and is substituted into the skin joint feature mathematical model. The measured data analysis table is shown in Table 1 and Table 2. From the table, it can be seen that the maximum system measurement error of the measured gap is 0.1483 mm, and the maximum system measurement error of the step difference is 0.0685 mm. In this embodiment, the maximum system measurement error is taken as the accuracy judgment standard, and the effective number 0.15 mm is taken as the gap error, and the effective number 0.07 mm is taken as the step difference error.

[0184] Table 1 Line structured light measurement gap result table (unit: mm)

[0185]

[0186] Table 2 Line structured light measurement step difference result table (unit: mm)

[0187]

[0188] Among them, the system measurement error is twice the standard deviation between the measured value and the true value. When there is no step difference, the point cloud data fluctuation is related to the quality of the line laser.

[0189] ②Face structured light vision measurement data acquisition:

[0190] In order to compare with the data measured by the line structured light, the butt joint standard specimen is also measured in the face structured light vision measurement sensor. The present application extracts 50 groups of data from the four kinds of joint point cloud data after cross section slicing, the extracted data corresponds to the detection area of the line structured light, and the joint feature extraction algorithm based on angle mutation is used to identify the joint, and is substituted into the skin joint feature mathematical model. The measured data analysis is shown in Table 3.

[0191] Table 3 Face structured light measurement gap result table (unit: mm)

[0192]

[0193]

[0194] Table 4 Face structured light measurement step difference result table (unit: mm)

[0195]

[0196] In the plane structure light measurement results table 4, the maximum system measurement error of the measured gap is 0.0968mm, the maximum system measurement error of the step difference is 0.1394mm, so the effective number 0.10mm is taken as the gap error, and the effective number 0.14mm is taken as the step difference error. Among them, the step difference is 0mm, the measured value represents the fluctuation of the plane structure light vision measurement point cloud data, which is related to the projected grating fringe, and the fluctuation value is higher than that of the line structure light measurement result, so when the step difference is greater than 0mm, the step difference measurement result is greatly affected. Among them, the system measurement error is twice the standard deviation between the measured value and the true value.

[0197] (4) Joint data alignment:

[0198] Aligning the measured data of the two kinds of structured light, the first laser stripe center pixel coordinate value of each joint type is obtained and the R, T matrix of the first group coordinate conversion, the specific parameter value is shown in table 5.

[0199] Table 5 data alignment parameter table

[0200]

[0201]

[0202] By solving the above parameters, the point cloud alignment diagram of each joint type can be obtained, as shown in Figure 15 .

[0203] (5) Joint data cleaning

[0204] In the part of rejecting abnormal values, set a = 0.05, and through the chi-square criterion table, G(n, a) = 2.956. By substituting the measured data into the chi-square criterion with the set critical value, it is proved that each group of data has no gross error.

[0205] (6) Joint data fusion

[0206] Then input the preprocessed data into the fusion algorithm. Since there are differences in the measurement of step difference and gap value between the two kinds of structured light measured data, the adaptive weighting coefficients of the joint features will be calculated respectively. The specific weighting coefficient table is shown in table 6.

[0207] Table 6 weighting coefficient parameter table

[0208]

[0209] Then the two kinds of measurement data are weighted and fused, and the fusion result is shown in Figure 16 , Figure 17 .

[0210] FromFigure 16 and Figure 17 It can be seen that, in the gap value fusion part, the fusion value is closer to the surface structure light measurement value; in the step difference value fusion part, the fusion value is closer to the line structure light measurement value. And compared with the two structure lights before fusion, the measurement value after fusion has smaller fluctuation, and the specific fusion result analysis is shown in Table 7 and Table 8.

[0211] Table 7 Gap value fusion result table (unit: mm)

[0212]

[0213] Table 8 Step difference value fusion result table (unit: mm)

[0214]

[0215] From the fusion result table, compared with the single structure light detection, the measurement mean value after fusion is closer to the theoretical value, and the mean error is smaller. And when the two measurement mean values are located on both sides of the true value, the fused result is better, wherein #2 gap, #3 gap, #4 gap and #1 step difference comply with this rule.

[0216] The maximum measurement error of the gap value system after fusion is 0.0887mm, and the maximum measurement error of the step difference value system is 0.0717mm. In this embodiment, the effective number 0.09mm is taken as the measurement error of the gap value after fusion, and the effective number 0.08mm is taken as the measurement error of the step difference value after fusion. From the fusion result, this fusion method balances the two structure light measurement results, improves the overall measurement precision, and verifies the effectiveness of the fusion method.

[0217] Example 2

[0218] In this embodiment, the joint measurement method of the application is used to measure the skin joint.

[0219] The application proposes two uses of this fusion method in actual measurement of the skin, i.e. detailed measurement and rough measurement.

[0220] (1) The object to be detected, such as a skin, is shown in Figure 18

[0221] (2) Detection method:

[0222] First, adjust the pose of each measurement sensor of the system to make the distance between the measurement sensor and the skin to be measured within the calibration range. After adjustment, start the comprehensive measurement system, first use the surface structure light vision measurement sensor to collect the skin joint, a total of 10 groups, as shown in Figure 19 The length of the joint area covered in the figure is 121.84mm.

[0223] As​Figure 20 As shown in the line structure light measurement part, in order to make the line structure light measurement part and the area structure light measurement part have an overlapping area, the embodiment adjusts the motor driver so that the laser line moves from the leftmost end of the area structure light measurement area to the direction shown in the figure, and when it moves to 30 mm away from the leftmost end of the area structure light measurement area, the motor stops running, and the camera is controlled to collect 10 groups of laser stripe images, which are named as position #1. Then the motor is started, and 10 groups of laser stripe data are collected at 60 mm and 90 mm away from the leftmost end of the area structure light measurement area, respectively, which are named as position #2 and position #3, respectively.

[0224] As described above, 10 groups of grating stripe images are collected in the area structure light vision measurement sensor, and 10 groups of laser stripe images are collected at three positions of the area structure light measurement area in the line structure light vision measurement sensor, a total of 30 groups of laser stripe images.

[0225] Note: ① Detailed measurement means corresponding the measured position of the line structure light laser line to the line point cloud of the area structure light measurement area, so as to perform high-precision detailed measurement.

[0226] ② Rough measurement means that the line structure light measurement results of multiple measurements are used to correct the area structure light measurement results in a small area, so that the overall step difference value and gap value measurement error is within the allowable range, and the overall measurement efficiency and accuracy are ensured, and the result reflects the change of the step difference value and the gap value of the entire measurement area.

[0227] (3) Joint measurement data acquisition

[0228] The joint features are extracted by using the joint feature recognition algorithm respectively, and are substituted into the joint feature skin joint feature mathematical model to obtain the measured data of the two kinds of structure light. The measured data of the three positions of the line structure light is shown in Table 9:

[0229] Table 9 Line structure light measurement data table

[0230] Position Line-Gap Line-Step Position #1 2.5749 0.7263 Position #2 2.4392 0.7259 Position #3 2.3941 0.7107

[0231] The measured data of the area structure light is shown in Table 9. Figure 21

[0232] (4) Joint data alignment

[0233] The measured data of the two kinds of structure light are aligned to obtain the first laser stripe center pixel coordinate value and the R, T matrix of the first group of coordinate conversion, which is one of the data alignment parameter tables, as shown in Table 10.

[0234] Table 10 Data alignment parameter table ​

[0235]

[0236] As Figure 22 shown, this is a set of point cloud alignment charts after preprocessing of the first-level fusion data.

[0237] (5) Joint data cleaning

[0238] In the outlier rejection part, set a = 0.05, and G(n, a) = 2.956 can be obtained by checking the Grubbs criterion table. By substituting the measured data into the Grubbs criterion with the set critical value, it is proved by experiments that each group of data has no gross error.

[0239] (6) Joint data fusion

[0240] After data alignment and data cleaning of each group of line structured light and area structured light measurement, substitute each group of data into the second-level fusion algorithm to calculate the mean and variance, and thus obtain the weighted coefficients of the line and area structured light vision measurement sensor, as shown in Table 11.

[0241] Table 11 Weighted coefficient table

[0242]

[0243] Finally, the fusion values of the measured 3 positions of the line structured light and the area structured light are obtained, and the fusion results are shown in Figure 23 .

[0244] As Figure 23 can be seen, the gap value fusion results are all within the range of 2.42 mm to 2.58 mm, and the step difference value fusion results are all within the range of 0.66 mm to 0.78 mm.

[0245] When measuring the skin:

[0246] ① Detailed measurement

[0247] The specific detailed measurement results are shown in Table 12, which shows the measured mean value of the fusion results at the specific position.

[0248] Table 12 Skin measurement result analysis table (unit: mm)

[0249]

[0250] In Table 10, the gap value measurement mean of the measured 3 positions of the skin is around 2.47 mm, the step difference value measurement mean is around 0.73 mm, the measurement results are relatively uniform, and the standard deviation of each measurement value is relatively constant, representing that the overall fluctuation is small.

[0251] ② Rough measurement

[0252] The measured values of the line structured light of positions #1, #2 and #3 are used to correct the measured values of the corresponding weighted coefficients of the measured area, wherein the measured values of position #1 are used to correct the area of 0mm to 40mm at the left end of the measured area of the distance plane structured light, the measured values of position #2 are used to correct the area of 40mm to 80mm at the left end of the measured area of the distance plane structured light, and the measured values of position #3 are used to correct the area of 80mm to 121.84mm at the left end of the measured area of the distance plane structured light, and the rough measurement result curve is as shown in Figure 24 The curve after sampling processing is shown in the figure.

[0253] From Figure 24 It can be seen that the curve after the fusion algorithm processing presents a segmented phenomenon, and the too low measurement value is lifted, and the too high measurement value is reduced, and on the basis of not changing the general trend of the curve in each area, the overall measurement result is balanced.

[0254] In summary, in the actual measurement of the skin joint, the detailed measurement result can be used to locate the measured area and view the result, and the more accurate measurement result can be obtained, and the rough measurement result can be used to observe the general measurement value change of the measured area, which is beneficial to the rapid and accurate measurement of the skin joint.

[0255] The line and plane structured light vision measurement technologies are creatively fused and applied to the skin joint measurement. The limitations of the two technologies are broken through, the overall measurement accuracy and measurement efficiency are improved, and a new idea is provided for the skin joint measurement.

[0256] The above-described embodiments are only the preferred specific implementation of the present application, and the protection scope of the present application is not limited thereto, and any simple change or equivalent replacement of the technical solutions within the technical range disclosed by the present application can be obtained by those skilled in the art, and all belong to the protection scope of the present application.

Claims

1. A skin seam measurement method based on data fusion, characterized in that: The following steps are involved: Obtain line structured light vision measurement point cloud data and surface structured light vision measurement point cloud data; Extracting line seam features from the line structured light visual measurement point cloud data; Splitting the discrete point cloud in the surface structured light vision measurement point cloud data into linear point clouds, and extracting edge points of the linear point clouds to obtain surface seam features; Substituting the line seam features and the surface seam features into the skin seam feature mathematical model respectively to obtain line seam optical measurement data and surface seam optical measurement data; Performing alignment processing on the line seam light measurement data and the surface seam light measurement data; An adaptive weighted fusion algorithm is used to fuse the aligned seam measurement data to obtain the seam step difference value and gap value.

2. The skin seam measurement method based on data fusion according to claim 1, characterized in that: The mathematical model of the skin seam characteristics is: Where ρ is the density of the seam point cloud, 1≤i≤n, n is the number of seam edge points, gap i is the distance from each edge point to the corresponding fitting line.

3. The skin seam measurement method based on data fusion according to claim 1, characterized in that: A line structured light vision measurement sensor and a surface structured light vision measurement sensor are used to collect line structured light vision measurement images and surface structured light vision measurement images respectively, and the line structured light vision measurement point cloud data and surface structured light vision measurement point cloud data are obtained by performing coordinate transformation processing on the line structured light vision measurement images and the surface structured light vision measurement images.

4. The skin seam measurement method based on data fusion according to claim 1, characterized in that: The laser stripes of the line structured light vision measurement point cloud data have four states in the pixel coordinate system, namely: with step difference and gap zero point, with step difference and no gap zero point, with no step difference and gap zero point, and with no step difference and no gap zero point.

5. The skin seam measurement method based on data fusion according to claim 4, characterized in that: Extracting line seam features from the line structured light visual measurement point cloud data using a seam feature recognition algorithm based on pixel deviation anomaly specifically includes the following steps: In view of the phenomenon of step difference and gap zero point, an improved slope algorithm is used to determine the critical points on both sides according to the calculated position of slope change; Where k(i) is the average slope, i is the row coordinate of the pixel, i ≥ 2, x(i-1), x(i+1), x(i-2), x(i+2) are the column coordinates of the pixels corresponding to i-1, i+1, i-2, i+2 respectively; For the phenomenon of zero points with step difference and no gap, and zero points with no step difference and no gap, the critical point is determined by searching for the break and missing of pixel column coordinates, and the Euclidean distance between two adjacent points is calculated to determine the edge points on both sides. Aiming at the phenomenon of zero points with no step difference and gaps, we first transform the zero points with no step difference and gaps into zero points with no step difference and no gaps. Then, we determine the critical points by searching for the breaks and missing of pixel column coordinates, calculate the Euclidean distance between two adjacent points, and determine the edge points on both sides.

6. The skin seam measurement method based on data fusion according to claim 5, characterized in that: A point cloud cross-section slicing method is used to split the discrete point cloud in the surface structured light vision measurement point cloud data into multiple linear point clouds, and a seam feature recognition algorithm based on angle mutation is used to extract edge points of the multiple linear point clouds to obtain surface seam features; The acquisition process of the multiple linear point clouds is as follows: Processing the surface structured light vision measurement point cloud data to obtain the height information topography of the data; In the height information topography, the y coordinate in the surface structured light vision measurement point cloud data is used as the feature for segmentation, and the point cloud data is sliced ​​with the XZ plane as the cross section to obtain a linear point cloud; The y pixel coordinates in the height information topography data are used as feature labels, and the data containing the same y coordinates are grouped together to obtain multiple linear height information topography images. By combining each set of data with the camera calibration parameters, multiple linear point clouds can be obtained.

7. The skin seam measurement method based on data fusion according to claim 6, characterized in that: The alignment processing of the line seam light measurement data and the surface seam light measurement data is specifically The following steps are involved: The first step is to obtain the center pixel coordinates of the laser stripe of the first laser stripe image of the line structured light measurement data; The second step is to align the obtained laser stripe center pixel coordinates with the height information matrix in the surface structured light measurement data; The third step is to use the point cloud cross-section slicing processing algorithm to obtain the point cloud coordinates of the first seam data measured by the line structured light corresponding to the surface structured light seam point cloud; In the fourth step, the pixel coordinate is increased by one, and steps 2 and 3 are repeated until the two structured light measurement data are completely consistent.

8. The skin seam measurement method based on data fusion according to claim 7, characterized in that: Before fusion, the aligned seam measurement data is cleaned to remove outliers. The specific steps are as follows: Step 1: Sort the n data collected by the line structured light vision measurement sensor and the surface structured light vision measurement sensor into x, i1 ,x i2 ,......,x in , where x in is the maximum value, x i1 is the minimum value; Step 2: Calculate the mean of the series With standard deviation s i ; Step 3: Calculate the statistics G of the maximum and minimum values max With G min ; Step 4: Set G(n,a) as the critical value, where a is the significance level. The specific G(n,a) can be obtained by determining the corresponding n and a and consulting the Grubbs criterion table. max ≥G min And G max >G(n,a), then x in is an outlier; when G min ≥G max And G min >G(n,a), then x i1 Outliers are then removed; Repeat steps 2 and 3 for the remaining data until no outliers appear, and the process of eliminating outliers is complete.

9. The skin seam measurement method based on data fusion according to claim 8, characterized in that: A distributed secondary data fusion algorithm is used to fuse the two aligned seam measurement data to obtain the fused seam step difference value and gap value, which specifically includes the following steps: Solving the directional variance of line structured light vision measurement sensors and surface structured light vision measurement sensors Calculate the weighting coefficients of the directions of the line structured light vision measurement sensor and the surface structured light vision measurement sensor; Where p = 1 or 2, Substitute the cleaned line seam light measurement data and surface seam light measurement data into the fusion value calculation formula to obtain the fusion result; Where, X r is the weighted fusion value, W p is the corresponding weighting coefficient, X p is the corresponding measurement value; p = 1 or 2; wherein X1 and X2 are the line seam light measurement data and the surface seam light measurement data after cleaning, and W1 and W2 are the corresponding weighting factors.

10. A skin seam measurement device based on data fusion, characterized in that: include: The line structured light vision measurement sensor and the surface structured light vision measurement sensor collect line structured light vision measurement images and surface structured light vision measurement images respectively; A data processing module is used to perform coordinate conversion processing on the line structured light vision measurement image and the surface structured light vision measurement image to obtain line structured light vision measurement point cloud data and surface structured light vision measurement point cloud data; The feature extraction module is used to perform feature recognition processing on the line structured light vision measurement point cloud data, perform cross-section processing on the surface structured light vision measurement point cloud data, and then perform feature recognition processing to extract line seam features and surface seam features; A data calculation module is used to substitute the line seam features and the surface seam features into the skin seam feature mathematical model to obtain the line seam light measurement data and the surface seam light measurement data; A data alignment module is used to align the line seam optical measurement data and the surface seam optical measurement data; The data fusion module is used to fuse the two aligned seam measurement data using an adaptive weighted fusion algorithm to obtain a fusion value.

Images