A method for extracting the center of a laser stripe on a highly reflective surface

Abstract

A method for extracting the center of a line laser stripe on a highly reflective surface. Median filtering and morphological filtering are performed on the light stripe image to reduce isolated noise and pseudo-stripe noise in the image; the light stripe image segmentation threshold is calculated to reduce the influence of noise with smaller grayscale values in the background on the center extraction; the region of interest of the light stripe image is determined to reduce the amount of computation required for the center extraction method; a back-propagation neural network is used to calculate the cross-sectional width of the light stripe in the image, the Hessian matrix parameters in the Steger algorithm are calculated based on the cross-sectional width, and the coordinates of the center point of the light stripe are roughly extracted; the coordinate continuity of the roughly extracted light stripe center is corrected to achieve smooth optimization of the light stripe center coordinates. The present invention alleviates the influence of image noise on the extraction of the light stripe center through image filtering and threshold segmentation, obtains the Hessian matrix parameters using the calculated light stripe width and roughly extracts the light stripe center, and smoothly optimizes the light stripe center coordinates based on the light stripe continuity, thereby meeting the requirements for stable extraction of the center of a line laser stripe on a highly reflective surface.

Description

Technical Field

[0001] The invention belongs to the technical field of center extraction of line laser stripes, and in particular relates to a method for center extraction of line laser stripes on a high-reflective surface. Background Art

[0002] With the continuous development of computer vision measurement technology, non-contact optical methods such as line laser measurement are increasingly being used in the aerospace field. During the measurement process of a line laser sensor, the center position of the laser stripe needs to be extracted based on the distribution of the laser stripe image, and the physical coordinates are determined based on the correspondence between the center position of the stripe and the pixel coordinates and physical coordinates. However, due to factors such as the surface roughness of the object to be measured, environmental noise, and exposure time, when measuring highly reflective metal surfaces, the proportion of specular reflected light on the surface of the workpiece to be measured is relatively large, resulting in overexposure and pseudo-fringe noise in some areas of the stripe image, which makes it difficult to extract the center of the laser stripe. Traditional methods for extracting the center of a laser stripe include the threshold method, the geometric center method, the extreme value method, the grayscale centroid method, the directional template method, and the Steger method (Hessian matrix method). Among them, the threshold method and the geometric center method require that the grayscale distribution of the stripe cross section approximately satisfies a Gaussian distribution. However, due to overexposure and noise interference, the grayscale distribution of the stripe image on a highly reflective surface does not satisfy a Gaussian distribution. The grayscale centroid method calculates the sub-pixel center coordinates of light stripes by weighted summation of grayscale values and pixel coordinates. This method performs poorly on images with large variations in light stripe curvature. The directional template method considers the curvature of the light stripe normal and extracts the light stripe center in the direction of the light stripe normal. However, its applicability is limited due to the limited number of directional angles in the template. The Steger method is a classic sub-pixel fringe center extraction algorithm that uses the Hessian matrix to extract the light stripe normal direction. This method can detect all light stripe normal directions and is highly applicable. However, its disadvantage is that the parameters in the Hessian matrix are easily affected by the light stripe width.

[0003] In recent years, relevant research has improved the traditional light stripe extraction method. In 2013, CN103400399B disclosed a line structured light center extraction method based on spatial moments. The edge of the laser line was detected by the Sobel operator, and the normal direction was calculated according to the gradient vector of the edge pixel point. The normal direction was divided into 8 directions at 45° intervals. The angle closest to the calculated normal direction was taken as the actual normal direction. The center coordinates were solved in the normal direction based on the spatial moment theory, and stable center coordinate extraction was achieved in the presence of overexposure. However, this method does not take into account the influence of pseudo-stripe noise in the image and the large normal direction intervals, and the light stripe extraction accuracy for some directions is insufficient. In 2020, CN111553874B disclosed a line structured light center extraction method for the surface of reflective nuclear fuel rods. The high-reflective area was removed by analyzing the grayscale characteristics of the light stripes. The center coordinates of the remaining light stripe areas were extracted using the adaptive row threshold method and the gradient centroid method. The least squares method was used to fit the light stripe center coordinates of the high-reflective area. This method has poor extraction effect on light strips with large curvature changes. At the same time, the fitted center coordinates of the high-reflective area are only applicable to continuous light strips. When the light strips in the high-reflective area are interrupted or the curvature changes greatly, the center coordinate extraction accuracy is insufficient.

[0004] Aiming at the problem that the traditional light stripe center extraction method has insufficient accuracy when measuring highly reflective surfaces, a new method for extracting the center of a line laser light stripe on a highly reflective surface based on the width and continuity of the light stripe is studied on the basis of the Steger method with strong applicability. Summary of the Invention

[0005] The main technical problem solved by the present invention is to overcome the shortcomings of the above methods. Aiming at the overexposure problem of line laser stripe images on highly reflective surfaces, a method for extracting the center of line laser stripes on highly reflective surfaces is invented.

[0006] The technical solution of the present invention:

[0007] A method for extracting the center of a laser stripe on a highly reflective surface is disclosed. The method first performs median filtering and morphological filtering on the stripe image to reduce isolated noise and pseudo-stripe noise in the stripe image. The method then calculates and determines a threshold for segmenting the stripe image to reduce the influence of background noise with low grayscale values on center extraction. The region of interest (ROI) of the stripe image is determined to reduce the computational complexity of the center extraction method. The back propagation neural network (BPNN) is used to calculate the cross-sectional width of the stripe in the image. The Hessian matrix parameters in the Steger algorithm are calculated based on the cross-sectional width and the coordinates of the stripe center are roughly extracted. The coordinate continuity of the roughly extracted stripe center is corrected to achieve smooth optimization of the stripe center coordinates. The method for extracting the center of a laser stripe on a highly reflective surface has the following specific steps:

[0008] (1) Light stripe image filtering

[0009] The overexposed light stripe image is processed by median filtering and morphological filtering. A window template of size n×n is set, and the grayscale value of the central pixel of the window template is replaced by the median grayscale value of other pixels in the window template. The entire image is filtered by sliding the window template. The light stripe image obtained after the median filtering is further subjected to morphological opening operation, that is, the light stripe image is first eroded and then dilated. The erosion operation is expressed as:

[0010] a(x,y)=min{f(x+x',y+y')-B(x',y')|(x',y')∈D b} (1)

[0011] Among them, a(x,y) is the light stripe image obtained by the corrosion operation, f(x,y) is the light stripe image before the corrosion operation, B(x',y') is the grayscale value corresponding to the structure element at the position (x',y'), and D b is the structural element area;

[0012] The expansion operation is expressed as:

[0013] z(x,y)=max{a(x-x',y-y')+B(x',y')|(x',y')∈D b} (2)

[0014] Among them, z(x,y) is the light stripe image obtained by the expansion operation;

[0015] (2) Image threshold segmentation

[0016] The optimal segmentation threshold of the filtered light stripe image is calculated using the maximum inter-class variance method; assuming that the optimal segmentation threshold of the light stripe image of size M×N is optimal threshold , the number and proportion of pixels belonging to the foreground of the image are recorded as N0 and ω0 respectively, the average gray value of the foreground is recorded as μ0, the number and proportion of pixels belonging to the background of the image are recorded as N1 and ω1 respectively, and the average gray value of the background is recorded as μ1; according to the mathematical relationship, we can get:

[0017]

[0018]

[0019] ω0+ω1=1 (5)

[0020] N0+N1=M×N (6)

[0021] The average grayscale value of the image is:

[0022] μ=ω0×μ0+ω1×μ1 (7)

[0023] The inter-class variance of the foreground and background grayscale of image segmentation is expressed as:

[0024] V=ω0×(μ0-μ) 2 +ω1×(μ1-μ) 2 (8)

[0025] The segmentation threshold that maximizes the inter-class variance is the optimal segmentation threshold for the image. threshold , the image is segmented by threshold using the truncation method, and the threshold segmentation result c(x,y) is expressed as:

[0026]

[0027] (3) Image ROI determination

[0028] Use the traversal method to search for the maximum grayscale value of each column of pixels in the light stripe image starting from the first column on the left side of the image, record the first column i1 and the last column i2 whose maximum grayscale value is not 0, i1 and i2 are respectively recorded as the left and right boundaries of the light stripe image ROI; for the part where the maximum grayscale value is greater than 0, the same traversal method is used to search from the top of the image for the position where the first pixel and the last pixel of each column are not 0, and the corresponding row numbers are recorded as j1 and j2, and the width is expanded by m pixels in the upper and lower directions. The upper and lower boundaries of the light stripe image ROI are (j1-m) and (j2+m), and the image ROI is finally determined to be (i1, i2, j1-m, j2+m);

[0029] (4) Rough extraction of light strip center coordinates

[0030] The BPNN is used to calculate the light stripe width. The input of BPNN is the grayscale vector of the light stripe image. First, the center point (x c ,y c ), that is, the xth c The first pixel position with grayscale value greater than 0 is recorded as m1, the last pixel position with grayscale value greater than 0 is recorded as m2, and the center coordinate is marked as y c =(m1+m2) / 2, take n pixels above and below the center point as the input layer vector of BPNN, the hidden layer of the network consists of p neurons, and the output is the width of the light strip cross section w c ;

[0031] Use the Hessian matrix to perform convolution operation with the light strip image:

[0032]

[0033]

[0034] Among them, g(x,y) is a two-dimensional Gaussian function with parameters r xx is the second-order derivative of the image in the x direction, r yy is the second-order derivative of the image in the y direction, r xy is the second-order mixed directional derivative of the image;

[0035] The eigenvector corresponding to the maximum eigenvalue of the matrix H(x,y) is denoted as (n x ,n y ), which is the normal direction of the light strip at that point in the image. The Taylor quadratic polynomial expansion of the grayscale distribution of the light strip is performed in the normal direction. The extreme point obtained by setting the first-order derivative to 0 is the center point of the light strip obtained by rough extraction (p x ,p y ):

[0036] (p x ,p y )=(x0+tn x ,y0+tn y ) (12)

[0037] Where (x0, y0) is the reference point in the light stripe image, and t is the calculated parameter:

[0038]

[0039] (5) Continuity correction of light strip center coordinates

[0040] The center point of the light strip obtained by rough extraction As the center of the circle, divide the coordinate neighborhood interval with a radius of K, and the coordinates of the other center points in the neighborhood interval are marked as P i (x i ,x i ), with the relative distance Δ to the center point at the center of the circle i As weighting factor for coordinate correction:

[0041]

[0042] The corrected coordinates of the center point of the light bar are expressed as:

[0043]

[0044] Traverse all the roughly extracted light bar center points to complete the continuity correction of the light bar center coordinates.

[0045] The beneficial effects of the present invention are as follows: median filtering and morphological filtering are performed on the light stripe image to reduce isolated noise and pseudo-stripe noise in the image; the light stripe image segmentation threshold is calculated to reduce the influence of noise with small grayscale value in the background on center extraction; the region of interest (ROI) of the light stripe image is determined to reduce the computational complexity of the center extraction method; the back propagation neural network (BPNN) is used to calculate the cross-sectional width of the light stripe in the image, the Hessian matrix parameters in the Steger algorithm are calculated according to the cross-sectional width, and the coordinates of the center point of the light stripe are roughly extracted; the coordinate continuity of the roughly extracted light stripe center is corrected to achieve smooth optimization of the light stripe center coordinates, thereby improving the stability of the center extraction of the laser light stripe on the highly reflective surface. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 This is a flow chart of the method for extracting the center of a line laser stripe on a highly reflective surface based on the width and continuity of the stripe.

[0047] Figure 2 It is a schematic diagram of the original light strip image.

[0048] Figure 3 This is a schematic diagram of the median filtering result.

[0049] Figure 4 This is a schematic diagram of the morphological filtering results.

[0050] Figure 5 This is a schematic diagram of the image threshold segmentation result.

[0051] Figure 6 It is a schematic diagram of the light stripe image ROI.

[0052] Figure 7 This is a schematic diagram of the BPNN structure.

[0053] Figure 8 This is a schematic diagram of the rough extraction results of the light strip center.

[0054] Figure 9 This is a schematic diagram of the correction results of the light bar center coordinates. DETAILED DESCRIPTION

[0055] The following describes the embodiments of the present invention in detail with reference to the accompanying drawings and technical solutions.

[0056] (1) Light stripe image filtering. The overexposed light stripe image is processed with a 5×5 median filter to reduce isolated noise in the image. The original light stripe image and the median filter result are as follows: Figure 2 and Figure 3 The image obtained by median filtering is further subjected to morphological opening operation. The shape of the structuring element of erosion and expansion is disk-shaped, and the size of the structuring element is set to 5. The morphological filtering result is as follows Figure 4 shown.

[0057] (2) Image threshold segmentation. The threshold of the filtered image is calculated using the maximum inter-class variance method, and the pixel values of the positions where the grayscale value is less than 89 in the light stripe image are set to 0, and the positions where the grayscale value is greater than 89 are not changed. The image after threshold segmentation is as follows: Figure 5 shown.

[0058] (3) Determine the image ROI. Use the traversal method to search for the maximum grayscale value of each pixel in the light stripe image starting from the first column on the left side of the image, and record the first column i1 and the last column i2 where the maximum grayscale value is not 0. i1 and i2 are respectively recorded as the left and right boundaries of the light stripe image ROI. For the part where the maximum grayscale value is greater than 0, the same traversal method is used to search from the top of the image for the position where the first pixel of each column is not 0 and the last pixel is not 0. The corresponding row numbers are recorded as j1 and j2. The width is expanded by 10 pixels in the upper and lower directions. The upper and lower boundaries of the light stripe image ROI are (j1-10) and (j2+10). Finally, the image ROI is determined to be (i1, i2, j1-15, j2+15). The divided light stripe image ROI is as follows: Figure 6 shown.

[0059] (4) Coordinate extraction of light stripe center. The input of BPNN is the grayscale vector of the light stripe image. First, the center point (x c ,y c ), that is, the xth c The first pixel position with grayscale value greater than 0 is recorded as m1, the last pixel position with grayscale value greater than 0 is recorded as m2, and the center coordinate is marked as y c =(m1+m2) / 2, 15 pixels are taken in the upper and lower directions of the center point as the input layer vector of BPNN, the hidden layer of the network consists of 10 neurons, and the output is the width of the light strip cross section. The BPNN structure is as follows Figure 7 As shown. The Hessian matrix parameter σ=6.0013 is calculated based on the average width of each cross-section light stripe, and the center coordinates of the light stripe are extracted in the normal direction. The rough extraction results of the center coordinates of the light stripe image are shown as follows. Figure 8 shown.

[0060] (5) Continuity correction of light strip center coordinates. The center point of the light strip obtained by rough extraction is used as the center of the circle, and the coordinate neighborhood interval with a radius of 4 is divided. The relative distance to the center point at the center of the circle is used as the weighting coefficient of the coordinate correction. The corrected coordinates of the light strip center are calculated. The correction results of the light strip center coordinates are as follows: Figure 9 shown.

[0061] The present invention provides a method for extracting the center of a line laser stripe on a highly reflective surface. The method can weaken the influence of high-reflective noise on the quality of the light stripe through image filtering and threshold segmentation. The light stripe center coordinates are roughly extracted by combining the light stripe width calculated by BPNN. The center coordinates are corrected according to the continuity of the light stripe. The light stripe center coordinates are smooth and highly adaptable, and can meet the requirements for high-precision extraction of the center of a line laser stripe on a highly reflective surface.

Claims

1. A method for extracting the center of a high-reflective surface laser stripe, characterized in that: Here are the steps: (1) Light stripe image filtering Perform median filtering and morphological filtering on the overexposed light stripe image; set a window template of size n×n, replace the grayscale value of the central pixel of the window template with the median grayscale value of other pixels in the window template, and filter the entire image by sliding the window template; The morphological opening operation is continued on the light stripe image obtained after the median filter, that is, the light stripe image is first eroded and then expanded. The erosion operation is expressed as: a(x,y)=min{f(x+x',y+y')-B(x',y')(x',y')∈D b } (1) Among them, a(x,y) is the light stripe image obtained by the corrosion operation, f(x,y) is the light stripe image before the corrosion operation, B(x',y') is the grayscale value corresponding to the structure element at the position (x',y'), and D b is the structural element area; The expansion operation is expressed as: z(x,y)=max{a(x-x',y-y')+B(x',y')(x',y')∈D b } (2) Among them, z(x,y) is the light stripe image obtained by the expansion operation; (2) Image threshold segmentation The optimal segmentation threshold of the filtered light stripe image is calculated using the maximum inter-class variance method; assuming that the optimal segmentation threshold of the light stripe image of size M×N is optimal threshold , the number and proportion of pixels belonging to the foreground of the image are recorded as N0 and ω0 respectively, the average gray value of the foreground is recorded as μ0, the number and proportion of pixels belonging to the background of the image are recorded as N1 and ω1 respectively, and the average gray value of the background is recorded as μ1; according to the mathematical relationship, we can get: ω0+ω1=1 (5) N0+N1=M×N (6) The average grayscale value of the image is: μ=ω0×μ0+ω1×μ1 (7) The inter-class variance of the foreground and background grayscale of image segmentation is expressed as: V=ω0×(μ0-μ) 2 +ω1×(μ1-μ) 2 (8) The segmentation threshold that maximizes the inter-class variance is the optimal segmentation threshold for the image. threshold , the image is segmented by threshold using the truncation method, and the threshold segmentation result c(x,y) is expressed as: (3) Image ROI determination Use the traversal method to search for the maximum grayscale value of each column of pixels in the light stripe image starting from the first column on the left side of the image, record the first column i1 and the last column i2 whose maximum grayscale value is not 0, i1 and i2 are respectively recorded as the left and right boundaries of the light stripe image ROI; for the part where the maximum grayscale value is greater than 0, the same traversal method is used to search from the top of the image for the position where the first pixel and the last pixel of each column are not 0, and the corresponding row numbers are recorded as j1 and j2, and the width is expanded by m pixels in the upper and lower directions. The upper and lower boundaries of the light stripe image ROI are (j1-m) and (j2+m), and the image ROI is finally determined to be (i1, i2, j1-m, j2+m); (4) Rough extraction of light strip center coordinates The BPNN is used to calculate the light stripe width. The input of BPNN is the grayscale vector of the light stripe image. First, the center point (x c ,y c ), that is, the xth c The first pixel position with grayscale value greater than 0 is recorded as m1, the last pixel position with grayscale value greater than 0 is recorded as m2, and the center coordinate is marked as y c =(m1+m2) / 2, take n pixels above and below the center point as the input layer vector of BPNN, the hidden layer of the network consists of p neurons, and the output is the width of the light strip cross section w c ; Use the Hessian matrix to perform convolution operation with the light strip image: Among them, g(x,y) is a two-dimensional Gaussian function with parameters r xx is the second-order derivative of the image in the x direction, r yy is the second-order derivative of the image in the y direction, r xy is the second-order mixed directional derivative of the image; The eigenvector corresponding to the maximum eigenvalue of the matrix H(x,y) is denoted as (n x ,n y ), which is the normal direction of the light strip at that point in the image. The Taylor quadratic polynomial expansion of the grayscale distribution of the light strip is performed in the normal direction. The extreme point obtained by setting the first-order derivative to 0 is the center point of the light strip obtained by rough extraction (p x ,p y ): (p x ,p y )=(x0+tn x ,y0+tn y ) (12) Where (x0, y0) is the reference point in the light stripe image, and t is the calculated parameter: (5) Continuity correction of light strip center coordinates The center point of the light strip obtained by rough extraction As the center of the circle, divide the coordinate neighborhood interval with a radius of K, and the coordinates of the other center points in the neighborhood interval are marked as P i (x i ,x i ), with the relative distance Δ to the center point at the center of the circle i As weighting factor for coordinate correction: The corrected coordinates of the center point of the light bar are expressed as: Traverse all the roughly extracted light bar center points to complete the continuity correction of the light bar center coordinates.

Images