Laser stripe center line extraction method based on adaptive exposure and pixel neighborhood

Abstract

The invention belongs to the technical field of optical three-dimensional measurement, and discloses a laser stripe center line extraction method based on adaptive exposure and pixel neighborhood, which comprises the steps of image preprocessing, adaptive exposure regulation and control, stripe image refinement and laser center line extraction. According to the method, a self-adaptive exposure regulation and control method is adopted in the image acquisition level, the quality of line laser image acquisition is improved, the extraction precision of the laser stripe center line is improved through a line laser refining method based on pixel neighborhood information, and more importantly, comprehensive optimization is carried out from the image acquisition level and the image processing level, so that the extraction efficiency is improved. And the extraction accuracy and robustness of laser stripe center line extraction are improved.

Description

Technical Field

[0001] The present application belongs to the technical field of optical three-dimensional measurement, and in particular to a method for extracting the center line of a laser stripe based on adaptive exposure and pixel neighborhood. Background Art

[0002] Optical three-dimensional measurement is widely used in the key dimension detection and shape measurement of parts in the automotive, aerospace, nuclear power and other fields in the industry. Active features are constructed by line lasers, and the laser stripe images collected by the camera are analyzed to reversely reconstruct the three-dimensional shape of the parts, and further measure and analyze the dimensions. Line laser three-dimensional measurement technology has the characteristics of non-contact, high precision, and high efficiency, which makes it have irreplaceable advantages in product manufacturing, quality assessment, and in-service inspection. Among them, the extraction of the center line of the laser stripe is one of the most critical technologies, but due to the influence of specific environments and working conditions, the laser stripe image will have excessive noise, uneven brightness, laser stripe faults and other phenomena. Therefore, a robust, high-precision, and high-efficiency laser stripe centerline extraction method can improve the measurement quality of parts.

[0003] At present, the mainstream methods for extracting the centerline of laser stripes include traditional methods and deep learning-based methods.

[0004] Traditional methods include extreme value method, grayscale centroid method, directional template method, and Steger method. Among them, the extreme value method and grayscale centroid method respectively extract the center line of the stripe by analyzing the gradient change and grayscale distribution of pixels, which have high computational efficiency but low extraction accuracy and are sensitive to the direction of the stripe; the directional template method pre-makes templates that adapt to different directions and then performs correlation analysis, which better solves the problem that the extraction is sensitive to the normal change of the laser stripe, but has the limitations of template selection and low accuracy; the Steger algorithm obtains the direction information of each point of the stripe by analyzing the image to the Hessian matrix, and then finds the minimum gradient point along the normal direction, which can adapt to the situation of large stripe curvature, has higher accuracy and wider adaptability, but because it needs to calculate the Hessian matrix of the entire image, the extraction efficiency is low.

[0005] The key to the extraction method based on deep learning lies in the preparation of training data. It is necessary to use high-precision centerline extraction results as training sets and test sets in advance. The weight information of the convolution kernel is obtained by training the global image or the area of ​​interest for actual extraction. This type of method has not yet been maturely applied in this field, and its stability, accuracy and efficiency need to be verified.

[0006] To sum up, firstly, the current mainstream methods only consider the grayscale of a single pixel and are easily affected by outliers. Secondly, the existing laser stripe centerline extraction methods do not adequately consider the acquisition method of high-quality images. Manual selection of camera exposure leads to large human error, low efficiency, and difficulty in dealing with unstructured scenes. Thirdly, image preprocessing is the primary step in stripe centerline extraction, and the current methods are also insufficient in accuracy and robustness. Summary of the invention

[0007] The purpose of the present application is to overcome the defects of the prior art, thereby providing a laser stripe centerline extraction method based on adaptive exposure and pixel neighborhood, thereby ensuring the accuracy and robustness of the adaptive extraction of the laser stripe centerline.

[0008] In order to achieve the above objectives, this application provides the following technical solutions:

[0009] A laser stripe centerline extraction method based on adaptive exposure and pixel neighborhood, comprising:

[0010] S1, the camera collects fringe images at the initial exposure time and pre-processes the collected images;

[0011] S2, for the stripe image after the contour is extracted in S1, the adaptive exposure time is calculated based on the stripe width and the grayscale cost, and S1 is cycled with the adaptive exposure time to obtain the optimal laser stripe image;

[0012] S3, performing stripe image refinement for the optimal laser stripe image obtained in S2;

[0013] S4. Based on the refined stripe image obtained in S3, the center line of the laser stripe is extracted.

[0014] According to one embodiment of the present application, S1 includes adaptive binarization of the stripe image and stripe contour extraction.

[0015] According to one embodiment of the present application, the image adaptive binarization step includes:

[0016] Divide the image into grids;

[0017] The OTSU adaptive binarization algorithm is performed on each block to adapt to the uneven illumination and background in different areas.

[0018] According to one embodiment of the present application, the stripe contour extraction step includes:

[0019] Connected domain extraction is performed on the binarized image to extract the contours of all connected domains containing stripes and other noises;

[0020] The roughly extracted contours are screened, and the contour area and the aspect ratio of the minimum envelope rectangle are used as screening conditions to obtain all the stripe contours and remove the noise area.

[0021] According to one embodiment of the present application, S2 includes:

[0022] S2.1: cost calculation of the stripe image after preliminary preprocessing;

[0023] S2.2: Iteratively update the exposure value based on cost minimization.

[0024] According to an embodiment of the present application, the cost function of adaptive exposure control is expressed as follows:

[0025]

[0026]

[0027] Where C(S) is the cost value corresponding to the set S of all laser stripe pixels in the image, C(S k ) is the laser stripe pixel subset S k The corresponding cost value is given by the width cost C h (S k ) and grayscale cost C g (S k ) weighted fusion, λ is the weight coefficient of the two, α(p ij ) is the width cost adaptive weight factor, β(p ij ) is the grayscale cost adaptive weight factor, h(p ij ) is the pixel p ij The corresponding fringe width, g(p ij ) is the pixel p ij The corresponding pixel gray value is For p ij The middle column of the row pixel, |·| is the absolute value, max(·) is the maximum value, h0 is the reference width, and g0 is the grayscale. Among them, the adaptive weight factor α(p ij ), β(p ij ) reflects the difference in width and grayscale distribution of line laser stripes. Considering that if the line laser is projected on a curved object, the stripe width changes with the change of pixels, for pixels with a sharp change in stripe width, its weight α(p ij ) is smaller. In addition, considering that stripes are usually composed of middle and high brightness areas and gradient areas on both sides, the closer the stripes are to the pixels on both sides, the greater the weight β(p ij ) will also decrease.

[0028] According to one embodiment of the present application, S2.2 includes:

[0029] Given an upper and lower limit of the exposure range (e min ,e max ), using a constant step size δ = (e max -e min ) / 20 method to calculate the cost within the exposure interval and update the exposure value until the exposure value that minimizes the cost is found;

[0030] In the neighborhood of this exposure value (e i -δ,e i +δ), and randomly generate exposure values ​​in the neighborhood. If the new exposure value is smaller than the current exposure value, the new exposure value is set to the current exposure value, and the neighborhood is updated to half of the original neighborhood. This process is repeated several times until the exposure value changes by an amount equal to e. i-1 -e i <ε;

[0031] The final stabilized exposure value is selected as the exposure value used for line laser stripe image acquisition to achieve adaptive exposure control.

[0032] According to an embodiment of the present application, a morphological operation is used in S3, and the stripe area is first corroded to remove fine noise and burrs, and then the stripe area is expanded to coarsen the stripe target to restore the original size and smooth the boundary area. A mask is generated based on the extracted stripe contour, and the mask ensures that when the center line is extracted, the pixel grayscale inside the contour is the original grayscale, and the pixel grayscale outside the mask is set to 0. The extracted stripe area is used as a mask, and the original pixels covered by the mask are Gaussian filtered.

[0033] According to an embodiment of the present application, in S3, the stripe regions of interest are segmented, and for the screened stripe contours, the minimum envelope rectangle is used as the region of interest, and all regions of interest are segmented from one image.

[0034] According to one embodiment of the present application, all points in the single point neighborhood are used in S4 to calculate the roughly extracted laser stripe grayscale centroid coordinates, and the calculation formula is as follows:

[0035]

[0036] Where w0 is the width of the neighborhood, h0 is the height of the neighborhood, is rounded down, g(i,j) is the pixel grayscale value of the point in the i-th row and j-th column of the image, u i is the x-coordinate of the center point of the i-th row of laser stripes.

[0037] According to one embodiment of the present application, the stripe normal calculation includes:

[0038] Use the roughly extracted initial neighborhood grayscale centroid point set Ω1 to calculate the normal at each point;

[0039] The normal at each point is calculated from the three adjacent points. The i-th point P i Normal direction n i By P i-1 ,P i ,P i+1 The three points are calculated as follows:

[0040] n i =k i / ||k i ||

[0041]

[0042] i=2,3,4,...,n-1

[0043] In the formula, k i is the i-th point P i The curvature at

[0044] The first point P1 and the last point P n The normal directions at are:

[0045] n1=(-τ 1y ,τ 1x )

[0046] n n =(-τ ny ,τ nx )

[0047] In the formula, τ n =(τ nx ,τ ny ) and τ1=(τ 1x ,τ 1y ) is the tangent direction at two points, and the calculation formula is as follows:

[0048]

[0049]

[0050] Obtain the normals of all points in the rough extracted initial neighborhood grayscale centroid point set, denoted as N1 = {n1, n2, n3, ..., n n}.

[0051] According to an embodiment of the present application, after rough extraction and normal calculation, the neighborhood grayscale centroid is refined along the normal, and the neighborhood information of the point is used instead of the single point information. i Normal direction n iTake a total of m points p1(x1,y1),p2(x2,y2),...,p m (x m ,y m ), then P i The grayscale centroid Q i (q ix ,q iy ) is calculated by the following formula:

[0052]

[0053]

[0054] Where w0 is the width of the neighborhood, h0 is the height of the neighborhood, To round down, g(y i ,x i ) is point p i The pixel gray value;

[0055] The grayscale centroid of the refined neighborhood of the laser stripe is calculated and recorded as Ω2 = {Q1, Q2, Q3, ..., Q n}.

[0056] Compared with the prior art, the laser stripe centerline extraction method based on adaptive exposure and pixel neighborhood provided in this application has the following beneficial effects:

[0057] This application not only adopts an adaptive exposure control method at the image acquisition level to improve the quality of line laser image acquisition, but also improves the extraction accuracy of the laser stripe centerline through a line laser refinement method based on pixel neighborhood information. More importantly, it comprehensively optimizes the image acquisition and image processing levels to improve the extraction accuracy and robustness of the laser stripe centerline extraction.

[0058] This application proposes an adaptive exposure control method at the level of line laser stripe image acquisition, constructs a cost function that fuses width cost and grayscale cost, and gives an exposure update method that fuses equal-step search and random search, avoiding the shortcomings of large human errors and low efficiency in manually setting exposure, and obtaining high-quality line laser stripe original images.

[0059] This application proposes a laser stripe centerline extraction algorithm that comprehensively considers pixel neighborhood information, stripe energy information, and stripe normal information at the centerline extraction level. The neighborhood of a single pixel is sampled using a sliding window, which makes full use of the neighborhood grayscale information and performs secondary extraction along the normal direction, ensuring efficiency while improving extraction accuracy.

[0060] In this application, at the image processing level, it is divided into two steps: image preprocessing and stripe image refinement. The former is embedded in the exposure adaptive control module, and the latter is embedded in the centerline extraction module. A laser stripe image processing method considering block binarization, stripe contour adaptive screening, morphological filtering, etc. is proposed to ensure the accuracy and robustness of stripe centerline extraction. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] In order to more clearly illustrate the technical solution of the present application, the following is a brief introduction to the drawings required for the technical description.

[0062] Figure 1 A flowchart of a laser stripe centerline extraction method based on adaptive exposure and pixel neighborhood provided in this application;

[0063] Figure 2 A schematic diagram of the overall algorithm flow provided for the embodiment of the present application;

[0064] Figure 3 A schematic diagram of the effect after binarization provided in an embodiment of the present application;

[0065] Figure 4 A schematic diagram of the effect after the morphological operation provided in the embodiment of the present application;

[0066] Figure 5 This is an example diagram of the results before the connected domain screening provided in the embodiment of the present application;

[0067] Figure 6 This is an example diagram of the results of connected domain screening provided in the embodiment of the present application;

[0068] Figure 7 A schematic diagram of the effect of adaptive exposure control provided in an embodiment of the present application;

[0069] Figure 8 A schematic diagram of a laser stripe centerline extraction method provided in an embodiment of the present application;

[0070] Fig. 9 This is a diagram showing the extraction effect of the center line of the laser stripe provided in an embodiment of the present application. DETAILED DESCRIPTION

[0071] The following is further explained in detail through specific implementation methods.

[0072] like Figure 1 As shown, the present application provides a laser stripe centerline extraction method based on adaptive exposure and pixel neighborhood, comprising:

[0073] S1: The camera collects fringe images at the initial exposure time and preprocesses the collected images;

[0074] S2: For the stripe image after the contour is extracted in S1, the adaptive exposure time is calculated based on the stripe width and grayscale cost, and S1 is cycled with the adaptive exposure time to obtain the optimal laser stripe image; taking the longitudinally distributed stripes as an example, the optimal laser stripe image is: the width of the bright area in the middle (grayscale value is 255) is about 3 pixels, and the grayscale of the pixels on both sides is in a gradual state, corresponding to the reference grayscale and reference width respectively;

[0075] S3: Refine the fringe image based on the optimal laser fringe image obtained in S2;

[0076] S4: Based on the refined stripe image obtained in S3, the center line of the laser stripe is extracted using the neighborhood-based secondary grayscale centroid method, taking into account the pixel neighborhood information, stripe energy information, and stripe normal information.

[0077] In S1, the collected fringe image is preprocessed including image adaptive binarization and fringe contour extraction.

[0078] Specifically, the image adaptive binarization step includes:

[0079] Divide the image into grids;

[0080] The OTSU adaptive binarization algorithm is performed on each block separately, collectively referred to as the block OTSU algorithm, to adapt to the uneven illumination and background in different areas.

[0081] Specifically, the stripe contour extraction step includes:

[0082] Connected domain extraction is performed on the binarized image to extract the contours of all connected domains containing stripes and other noises;

[0083] The roughly extracted contours are screened, and the contour area and the aspect ratio of the minimum envelope rectangle are used as screening conditions to obtain all the stripe contours and remove the noise area.

[0084] In S2, the adaptive exposure calculation includes the following two steps:

[0085] S2.1: cost calculation of the stripe image after preliminary preprocessing;

[0086] S2.2: Iteratively update the exposure value based on cost minimization.

[0087] In S2.1, the cost calculation of adaptive exposure control is a comprehensive cost calculation of stripe width and grayscale. The core of the grayscale centroid method is to use the energy distribution of the stripes, and the energy is reflected in the grayscale value in the image. Therefore, calculating the grayscale cost is part of the comprehensive cost; stripe width is also one of the important indicators. If the stripe width is too small, the grayscale gradient will be too extreme. If the stripe width is too wide, the proportion of high-energy pixels will be too high and the grayscale change will not be obvious. Both will deteriorate the extraction of the center line of the laser stripe.

[0088] Taking the longitudinal distribution of laser stripes as an example (similar cost functions can be derived for the transverse distribution of stripes), the cost function of adaptive exposure control can be expressed as follows:

[0089]

[0090]

[0091] Where C(S) is the cost value corresponding to the set S of all laser stripe pixels in the image, C(S k ) is the laser stripe pixel subset S k The corresponding cost value is given by the width cost C h (S k ) and grayscale cost C g (S k ) weighted fusion, λ is the weight coefficient of the two. Considering the heterogeneity of the width of the stripes when projected on the surface and the grayscale distribution of the stripes, the adaptive weight factor α(p ij ) and β(p ij ), which corresponds to the pixel p in the i-th row and j-th column ij To adapt to the differences in width and grayscale. ij ) and g(p ij ) represent pixel points p ij The corresponding stripe width and pixel gray value. Indicates p ij The middle column of the row pixel, |·| represents the absolute value, max(·) represents the maximum value, h0 and g0 are the reference width and grayscale. Among them, the adaptive weight factor α(p ij ), β(p ij ) reflects the difference in width and grayscale distribution of line laser stripes. Considering that if the line laser is projected on a curved object, the stripe width changes with the change of pixels, for pixels with a sharp change in stripe width, its weight α(p ij ) is smaller. In addition, considering that stripes are usually composed of middle and high brightness areas and gradient areas on both sides, the closer the stripes are to the pixels on both sides, the more their weight β(p ij) is smaller. Based on this, the comprehensive cost value corresponding to the line laser stripe image can be calculated to provide a basis for the iterative update of the exposure value.

[0092] In S2.2, the iterative update of adaptive exposure control first gives an upper and lower limit of the exposure range (e min ,e max ), first use the same step size δ=(e max -e min ) / 20 in the exposure interval, and update the exposure value until the exposure value that minimizes the cost is found. i -δ,e i +δ), and randomly generate exposure values ​​in the neighborhood. If the new exposure value is smaller than the current exposure value, the new exposure value is set to the current exposure value, and the neighborhood is updated to half of the original neighborhood. This process is repeated several times until the exposure value changes by an amount equal to e. i-1 -e i <ε. The final stable exposure value is selected as the exposure value used for line laser stripe image acquisition to achieve adaptive exposure control.

[0093] In S3, stripe image refinement includes stripe mask generation, region of interest segmentation, morphological operations, and Gaussian filtering.

[0094] In S3, mask generation is to generate a mask from the extracted stripe profile to provide a pixel area for subsequent laser stripe centerline extraction. The mask ensures that the subsequent laser stripe extraction only considers the pixels inside the mask, while the pixels outside the mask are set to 0.

[0095] In S3, the stripe region of interest is segmented for the screened stripe contour, and its minimum envelope rectangle is used as the region of interest. All regions of interest are segmented from one image to prepare for the subsequent centerline extraction.

[0096] In S3, the image morphological operation adopts the opening operation, firstly, the stripe area is corroded to remove small noise and burrs, and then the stripe area is expanded to coarsen the target stripes and restore them to their original size with smooth boundaries. Finally, the stripe image is Gaussian filtered, and the extracted stripe area is used as a mask to perform Gaussian filtering on the original pixels covered by the mask.

[0097] In S4, the neighborhood-based secondary grayscale centroid method is used to extract the center line of the laser stripe, while considering the pixel neighborhood information, stripe energy information and stripe normal information. In pixel sampling, the neighborhood of the sliding window sampling pixel is considered to make full use of all pixels in the neighborhood; the energy distribution of the stripe is reflected in the grayscale distribution, so the grayscale centroid method is still used; the grayscale centroid can only be extracted in the horizontal or vertical direction. Based on the initially extracted grayscale centroid point set, the grayscale centroid point is further refined along the normal of each point to improve the extraction accuracy.

[0098] The technical solution of the present application abandons the traditional idea of ​​sampling the grayscale of a single point, and then uses a sliding window to sample the neighborhood of each pixel on the stripe, and improves the traditional grayscale centroid to the neighborhood grayscale centroid, which means calculating the grayscale centroid along the horizontal direction (assuming that the stripe is in the vertical direction, if the stripe is in the horizontal direction, the vertical direction is selected here), but does not directly use the pixel value of a single point, but uses the average pixel of all points in the neighborhood of the single point. The specific calculation method is as follows:

[0099]

[0100] Where w0 and h0 are the width and height of the neighborhood (take an odd number), represents rounding down, g(i,j) represents the pixel grayscale value of the point in the i-th row and j-th column of the image, u i is the x-coordinate of the center point of the i-th row of laser stripes.

[0101] At the same time, the normal information of the stripes is considered. Based on the point set calculated by the initial neighborhood grayscale centroid, the curvature at each point is calculated, and then the normal at each point is calculated. Then the second refined neighborhood grayscale centroid is calculated along the normal of the initial center line point set, and the second finely extracted point set is obtained as the center line of the laser stripes. The extraction of the second refined neighborhood grayscale centroid still follows the same formula, and the direction of the sliding window sampling can be changed to the normal direction.

[0102] Example

[0103] like Figures 2 to 8 As shown, this embodiment adopts a laser stripe centerline extraction method based on adaptive exposure and pixel neighborhood, which includes image preprocessing, adaptive exposure control, stripe image refinement, and laser stripe centerline extraction. The first two steps are continuously cycled until the laser stripe image under the best exposure is found, and then the latter two steps are implemented. The specific implementation steps of this method include:

[0104] Step 1: Fringe image processing.

[0105] (1) Image Binarization

[0106] The original image is binarized and the pixels are classified into two categories: 0 and 255, which is convenient for subsequent morphological operations and connected domain analysis. The binarization adopts the block OTSU strategy to divide the image into different regions. The OTSU binarization algorithm is performed in each region, that is, each region uses the binarization threshold that best suits itself. It can effectively deal with the interference of uneven lighting and background, making the binarization more robust.

[0107] (2) Stripe contour extraction

[0108] For the binarized image, we first extract the connected domains, and then calculate the area and minimum envelope rectangle of all the extracted connected domains. Through two-layer screening, we get the stripe connected domains, and then remove the noise and other objects. The screening conditions are as follows:

[0109] S>S0

[0110] H / W>σ

[0111] Where S is the area of ​​the connected domain, S0 is the area threshold given in advance, H and W are the height and width of the minimum envelope rectangle of the connected domain, and σ is the height-to-width ratio given in advance.

[0112] Among them, the first level of screening is mainly for small noise connected domains. Since they usually occupy a small number of pixels, they can be screened out by area size. The second level of screening is the aspect ratio of the minimum envelope rectangle. Since the stripes are assumed to be in the vertical direction, the aspect ratio of the stripes is usually large. The second level of screening can be used to remove the remaining objects and retain the stripe connected domain. It should be noted here that if the stripes are in the horizontal direction, the second level of screening should be corrected to W / H>σ, that is, screening based on the aspect ratio.

[0113] Examples of results before screening include Figure 5 As shown, the results after screening are as follows Figure 6 As shown, it can be seen that the block and point noise around the line laser stripes are removed after screening.

[0114] (3) Stripe mask generation

[0115] The stripe contour extracted in the previous step is used to make a mask, so that the pixels inside the stripe contour maintain the original grayscale, and the pixels outside the stripe contour are set to 0, so that the subsequent image morphological operations, filtering and laser stripe centerline extraction are not affected by the noise and foreign matter areas outside the stripe contour.

[0116] (4) Region of Interest Segmentation

[0117] On the basis of the minimum envelope rectangle of the stripe connected domain, several pixels are expanded outward to ensure that all the original stripes are included, and several stripe regions of interest are segmented for the subsequent extraction of the center line of the laser stripe.

[0118] (5) Morphological filtering

[0119] For the selected stripe connected domain, morphological operation is performed first, and then Gaussian filtering is performed.

[0120] In this embodiment, the image morphological operation selects the opening operation. First, the image is corroded to remove the relatively small noise points and connecting lines in the image, so that the stripe connected domain is reduced and refined. Then the image is expanded to enlarge and coarsen the stripe connected domain. After the opening operation, the burrs around the stripe connected domain will be removed, and the boundary area will be smooth, which is more conducive to the accurate extraction of the center line of the laser stripe.

[0121] Gaussian filtering is not performed on the binary image, but with the stripe connected domain extracted as a mask, and the pixel grayscale value of the original image is used to perform Gaussian filtering in the area covered by the connected domain to prepare for centerline extraction.

[0122] Step 2: Adaptive exposure bar adjustment.

[0123] (1) Cost function calculation

[0124] The cost calculation of adaptive exposure control is a comprehensive cost calculation of stripe width and grayscale. The core of the grayscale centroid method is to use the energy distribution of the stripes, and the energy is reflected in the grayscale value in the image. Therefore, calculating the grayscale cost is part of the comprehensive cost; stripe width is also one of the important indicators. If the stripe width is too small, the grayscale gradient will be too extreme. If the stripe width is too wide, the proportion of high-energy pixels will be too high and the grayscale change will not be obvious. Both will deteriorate the extraction of the center line of the laser stripe.

[0125] Taking the longitudinal distribution of laser stripes as an example (similar cost functions can be derived for the transverse distribution of stripes), the cost function of adaptive exposure control can be expressed as follows:

[0126]

[0127] in,

[0128]

[0129] Where C(S) is the cost value corresponding to the set S of all laser stripe pixels in the image, C(S k ) is the laser stripe pixel subset S k The corresponding cost value is given by the width cost C h (S k ) and grayscale cost C g (S k) weighted fusion, λ is the weight coefficient of the two. Considering the heterogeneity of the width of the stripes when projected on the surface and the grayscale distribution of the stripes, the adaptive weight factor α(p ij ) and β(p ij ), which corresponds to the pixel p in the i-th row and j-th column ij To adapt to the differences in width and grayscale. ij ) and g(p ij ) represent pixel points p ij The corresponding stripe width and pixel gray value. Indicates p ij The middle column of the row pixel, |·| represents the absolute value, max(·) represents the maximum value, h0 and g0 are the reference width and grayscale. Based on this, the comprehensive cost value corresponding to the line laser stripe image can be calculated, providing a basis for the iterative update of the exposure value.

[0130] (2) Exposure Update

[0131] First, give an upper and lower limit of the exposure range (e min ,e max ), first use the same step size δ=(e max -e min ) / 20 in the exposure interval, and update the exposure value until the exposure value that minimizes the cost is found. i -δ,e i +δ), and randomly generate exposure values ​​in the neighborhood. If the new exposure value is smaller than the current exposure value, the new exposure value is set to the current exposure value, and the neighborhood is updated to half of the original neighborhood. This process is repeated several times until the exposure value changes by an amount equal to e. i-1 -e i <ε. The final stable exposure value is selected as the exposure value used for line laser stripe image acquisition to achieve adaptive exposure control.

[0132] Step 3: Extract the center line of the laser stripe.

[0133] (1) Grayscale centroid calculation based on neighborhood information

[0134] The pixel neighborhood information, stripe energy information and stripe normal information are considered at the same time. This embodiment abandons the traditional idea of ​​sampling the grayscale of a single point, and then uses a sliding window to sample the neighborhood of each pixel on the stripe, and improves the traditional grayscale centroid to the neighborhood grayscale centroid, which means calculating the grayscale centroid along the horizontal direction (assuming that the stripe is in the vertical direction, if the stripe is in the horizontal direction, the vertical direction is selected here), but does not directly use the pixel value of a single point, but uses the average pixel of all points in the neighborhood of the single point. The specific calculation method is as follows:

[0135]

[0136] Where w0 and h0 are the width and height of the neighborhood (take an odd number), represents rounding down, g(i,j) represents the pixel grayscale value of the point in the i-th row and j-th column of the image, u i is the x-coordinate of the center point of the i-th row of laser stripes.

[0137] Finally, point P i (u i ,i) is the initial neighborhood grayscale centroid of the i-th row of stripe. All grayscale centroids on the stripe constitute the initial neighborhood grayscale centroid point set of rough extraction, denoted as Ω1={P1,P2,P3,...,P n Using neighborhood information instead of single point information can maximize the use of neighborhood group information rather than individual information of a single point, which significantly improves the accuracy compared to the traditional grayscale centroid method.

[0138] (2) Fringe normal calculation

[0139] The normal at each point is calculated using the roughly extracted initial neighborhood grayscale centroid point set Ω1. Specifically, the normal at each point is calculated from the three adjacent points, and the i-th point P i Normal direction n i By P i-1 ,P i ,P i+1 The three points are calculated as follows:

[0140] n i =k i / ||k i ||

[0141]

[0142] i=2,3,4,...,n-1

[0143] In the formula, k i is the i-th point P i The curvature of the .

[0144] The first point P1 and the last point P n The normal direction at is calculated as follows. The normal directions of the two are:

[0145] n1=(-τ 1y ,τ 1x )

[0146] n n =(-τ ny ,τ nx )

[0147] In the formula, τ n =(τ nx ,τ ny ) and τ1=(τ 1x ,τ 1y ) is the tangent direction at the two points and can be calculated as follows:

[0148]

[0149]

[0150] At this point, all the normal directions of the initial neighborhood grayscale centroid point set roughly extracted have been obtained, denoted as N1 = {n1, n2, n3, ..., n n}.

[0151] Step 4: Refine the grayscale centroid along the normal direction.

[0152] After rough extraction and normal calculation, the neighborhood grayscale centroid can be refined along the normal, and the neighborhood information of the point is still used instead of the single point information. Suppose along the i-th point P i Normal direction n i Take m points in total Then P i The grayscale centroid Q i (q ix ,q iy ) can be calculated by the following formula:

[0153]

[0154]

[0155] Where w0 and h0 are the width and height of the neighborhood (take an odd number), Indicates rounding down, g(y i ,x i ) represents point p i So far, the grayscale centroids of the refined neighborhood of the laser stripe have been fully calculated and recorded as Ω2 = {Q1, Q2, Q3, ..., Q n}. This point set is used as the final extracted laser stripe center line point set. The extraction results are as follows: Fig. 9 shown.

[0156] The above description is only a specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by any technician familiar with the technical field within the technical scope disclosed in the present application should be covered within the protection scope of the present application.

Claims

1. A laser stripe centerline extraction method based on adaptive exposure and pixel neighborhood, characterized in that: include: S1, the camera collects fringe images at the initial exposure time and pre-processes the collected images; S2, for the stripe image after the contour is extracted in S1, the adaptive exposure time is calculated based on the stripe width and the grayscale cost, and S1 is cycled with the adaptive exposure time to obtain the optimal laser stripe image; S3, performing stripe image refinement for the optimal laser stripe image obtained in S2; S4. Based on the refined stripe image obtained in S3, the center line of the laser stripe is extracted.

2. The laser stripe centerline extraction method based on adaptive exposure and pixel neighborhood according to claim 1, characterized in that: S1 includes: (1) Adaptive Binarization of Fringe Images Divide the image into grids; The OTSU adaptive binarization algorithm is performed on each block to adapt to the uneven illumination and background in different areas. (2) Stripe contour extraction Connected domain extraction is performed on the image after adaptive binarization to extract the contours of all connected domains containing stripes and other noises; The roughly extracted contours are screened, and the contour area and the aspect ratio of the minimum envelope rectangle are used as screening conditions to obtain all the stripe contours and remove the noise area.

3. The laser stripe centerline extraction method based on adaptive exposure and pixel neighborhood according to claim 1, characterized in that S2 include: S2.1: cost calculation of the stripe image after preliminary preprocessing; S2.2: Iteratively update the exposure value based on cost minimization.

4. The laser stripe centerline extraction method based on adaptive exposure and pixel neighborhood according to claim 3, characterized in that: In S2.1, the cost function of adaptive exposure control is expressed as follows: Where C(S) is the cost value corresponding to the set S of all laser stripe pixels in the image, C(S k ) is the laser stripe pixel subset S k The corresponding cost value is given by the width cost C h (S k ) and grayscale cost C g (S k ) weighted fusion, λ is the weight coefficient of the two, α(p ij ) is the width cost adaptive weight factor, β(p ij ) is the grayscale cost adaptive weight factor, h(p ij ) is the pixel p ij The corresponding fringe width, g(p ij ) is the pixel p ij The corresponding pixel gray value is For p ij The middle column of the row pixel, |·| is the absolute value, max(·) is the maximum value, h0 is the reference width, and g0 is the grayscale.

5. The laser stripe centerline extraction method based on adaptive exposure and pixel neighborhood according to claim 3, characterized in that: S2.2 includes: Given an upper and lower limit of the exposure range (e min ,e max ), using a constant step size δ = (e max -e min ) / 20 method to calculate the cost within the exposure interval and update the exposure value until the exposure value that minimizes the cost is found; In the neighborhood of this exposure value (e i -δ,e i +δ), and randomly generate exposure values ​​in the neighborhood. If the new exposure value is smaller than the current exposure value, the new exposure value is set to the current exposure value, and the neighborhood is updated to half of the original neighborhood. This process is repeated several times until the exposure value changes by an amount equal to e. i-1 -e i <ε; The final stabilized exposure value is selected as the exposure value used for line laser stripe image acquisition to achieve adaptive exposure control.

6. The method for extracting the center line of a laser stripe based on adaptive exposure and pixel neighborhood according to claim 1, characterized in that: S3 uses morphological operations to first perform an erosion operation on the stripe area to remove small noise and burrs, and then perform an expansion operation on the stripe area to coarsen the stripe target and restore it to its original size, and smooth the boundary area. The extracted stripe area is used as a mask, and Gaussian filtering is performed on the original pixels covered by the mask.

7. The laser stripe centerline extraction method based on adaptive exposure and pixel neighborhood according to claim 1, characterized in that: In S3, the stripe regions of interest are segmented. For the screened stripe contours, the minimum envelope rectangle is used as the region of interest, and all regions of interest are segmented from one image.

8. The laser stripe centerline extraction method based on adaptive exposure and pixel neighborhood according to claim 1, characterized in that: In S4, the coordinates of the grayscale centroid of the roughly extracted laser stripes are calculated using the following formula: Where w0 is the width of the neighborhood, h0 is the height of the neighborhood, is rounded down, g(i,j) is the pixel grayscale value of the point in the i-th row and j-th column of the image, u i is the x-coordinate of the center point of the i-th row of laser stripes.

9. The laser stripe centerline extraction method based on adaptive exposure and pixel neighborhood according to claim 1, characterized in that: In S4, the stripe normal calculation includes: Use the roughly extracted initial neighborhood grayscale centroid point set Ω1 to calculate the normal at each point; The normal at each point is calculated from the three adjacent points. The i-th point P i Normal direction n i By P i-1 ,P i ,P i+1 The three points are calculated as follows: n i =k i / ||k i || i=2,3,4,...,n-1 In the formula, k i is the i-th point P i The curvature at The first point P1 and the last point P n The normal directions at are: n1=(-τ 1y ,t 1x ) n n =(-τ ny ,t nx ) In the formula, τ n =(τ nx ,τ ny ) and τ1=(τ 1x ,τ 1y ) is the tangent direction at two points, and the calculation formula is as follows: Obtain the normals of all points in the rough extracted initial neighborhood grayscale centroid point set, denoted as N1 = {n1, n2, n3, ..., n n }.

10. The method for extracting the center line of a laser stripe based on adaptive exposure and pixel neighborhood according to claim 9, characterized in that: After rough extraction and normal calculation, the grayscale centroid of the neighborhood is refined along the normal, and the neighborhood information of the point is used instead of the single point information. i Normal direction n i Take a total of m points p1(x1,y1),p2(x2,y2),...,p m (x m ,y m ), then P i The grayscale centroid Q i (q ix ,q iy ) is calculated by the following formula: Where w0 is the width of the neighborhood, h0 is the height of the neighborhood, To round down, g(y i ,x i ) is point p i The pixel gray value; The grayscale centroid of the refined neighborhood of the laser stripe is calculated and recorded as Ω2 = {Q1, Q2, Q3, ..., Q n }.

Images