Laser spot array detection method and system for structured light parameter calibration

Abstract

The invention discloses a laser spot array detection method and system for structured light parameter calibration, and the method comprises the steps: capturing a laser array spot image on a laser calibration plane through a structured light camera, and carrying out the distortion correction processing and image filtering; extracting a red channel image, performing full-image adaptive threshold segmentation, corrosion and expansion operations, and screening Gaussian centroid coordinates and corresponding channels of candidate regions according to region areas; calculating the position of a peak value according to the position relation and the energy distribution condition of the candidate points, positioning the position of the light spot array, and generating a theoretical light spot array; calculating an actual light spot position by utilizing projective transformation and complementing missing light spots, carrying out secondary positioning correction on a light spot array by utilizing a seed point segmentation algorithm, and iteratively carrying out projective transformation to correct light spot deviation to obtain a sub-pixel-level light spot position; according to the invention, light spot sub-pixel position detection in structured light parameter calibration is realized, the difficulty of structured light camera pipeline calibration is reduced, and the reliability and robustness are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of laser array spot detection, and more particularly to a laser spot array detection method and system for structured light parameter calibration. Background Art

[0002] At present, industrial endoscopes based on structured light have been widely used in defect detection and measurement of industrial products. However, when using structured light for measurement, the structured light measurement system needs to be calibrated in advance, including camera calibration and structured light parameter calibration. Structured light parameter calibration refers to determining the projection parameters of structured light in the camera coordinate system, so as to determine the three-dimensional spatial coordinates of the structured light stripes or spots by combining the image position of the structured light on the surface of the object during measurement.

[0003] Laser spot array is a projection mode often used in structured light. Therefore, determining the precise position of the laser spot array in the camera image and identifying the number of the spot in the array are important links in achieving structured light positioning and structured light measurement tasks.

[0004] However, in industrial endoscope structured light measurement, due to the limitation of the endoscope pipeline size, the angle between the camera axis direction in the endoscope and the main projection direction of the laser is small and close to parallel, and the baseline distance between the center of the camera lens and the center of the laser projection light source is very short. Therefore, the working distance of the industrial endoscope is very small, which leads to the following problems in laser spot detection:

[0005] (1) After the laser light spot array is reflected on the surface of the object, part of the light is reflected again by the surface of the endoscope lens to the surface of the object to form interference light spots, which may cause more light spots than the actual number of light spot arrays to appear on the image; (2) The uneven intensity of the laser light spot projection causes some light spots to appear with a white center and a diffuse area around them, while some light spots only have the corresponding laser color without a white center. In addition, the light spot intensity may be insufficient to form an effective light spot image. In addition, the different degrees of absorption or reflection of the laser on the surfaces of different objects will also cause great differences in the laser spot image.

[0006] Therefore, how to provide a laser spot array detection method for structured light parameter calibration in a structured light endoscope system to improve the robustness and reliability of the structured light parameter calibration, while improving the accuracy and speed of the structured light pipeline calibration, is a problem that technical personnel in this field urgently need to solve. Summary of the invention

[0007] In view of this, the present invention provides a laser spot array detection method and system for structured light parameter calibration to solve some of the technical problems mentioned in the background technology.

[0008] In order to achieve the above object, the present invention adopts the following technical solution:

[0009] A laser spot array detection method for structured light parameter calibration comprises the following steps:

[0010] S1. Use a structured light camera to capture a full-color image to be detected containing an M×N array of laser spots on a laser calibration plane, and use the internal lens distortion coefficient of the calibrated structured light camera to perform distortion correction processing on the image to be detected to obtain a corrected image, and perform image filtering processing on the corrected image to reduce image noise;

[0011] S2. decompose the filtered image into RGB three-channel images, extract the red channel, and filter out the interference of other channel colors on spot detection;

[0012] S3. Perform full-image adaptive threshold segmentation on the red channel image, perform erosion and dilation operations on the segmented image, and perform regional evaluation on each connected domain, sort by area and screen the Gaussian centroid coordinates and corresponding channels of the first 2×M×N candidate regions;

[0013] S4. Using the position relationship and grayscale distribution of the candidate points, perform energy integration operation and calculate the peak position, locate the position of the spot array and extract the center position of the M×N spot array, and generate a theoretical spot array with equidistant distribution;

[0014] S5. Using the position of the theoretical light spot array, the actual light spot position is calculated by projective transformation, and the missing light spots are completed. The seed point segmentation algorithm is used to perform secondary positioning correction on the light spot array, and the projective transformation is iteratively performed to correct the light spot deviation. Finally, the straight line is fitted to obtain the sub-pixel light spot position.

[0015] Preferably, the image filtering process is specifically to successively perform 5×5 Gaussian filtering, mean filtering, and 3×3 Gaussian filtering.

[0016] Preferably, the method for performing regional evaluation on each connected domain is:

[0017]

[0018] Among them: r means passing the evaluation system, b means the indicator exceeds the maximum threshold, and s means the indicator is below

[0019] Minimum threshold, is the connected domain Ω i area max 、area min are the upper and lower bounds of the area of ​​the connected domain; is the connected domain Ω i The aspect ratio of the bounding rectangle, ratiomax 、ratio min are the upper and lower bounds of the aspect ratio of the circumscribed rectangle; is the intensity of the red channel at the Gaussian centroid of the connected domain; R the It is the minimum threshold of the intensity of the red channel at the center of mass of the Gaussian spot.

[0020] Preferably, step S4 specifically includes:

[0021] S41. Using the maximum area of ​​the candidate region, the maximum radius of the spot is calculated, and the Gaussian centroid coordinates and the maximum radius of the spot are used to generate a template for the entire image, that is, for each Gaussian centroid coordinate, a circular MASK segmentation image is generated with the maximum radius of the spot as the radius;

[0022] S42. Integrate the segmented image about the X and Y axes respectively to obtain two one-dimensional results ΔX and ΔY after integration;

[0023] S43. Use the peak values ​​of the two integrated one-dimensional results to locate the light spot, and use the four positions of the first peak position ΔX(1) and the Mth peak position ΔX(M) in ΔX and the first peak position ΔY(1) and the Nth peak position ΔY(N) in ΔY to locate the four corner coordinates of the light spot array (ΔX(1), ΔY(1)), (ΔX(1), ΔY(N)), (ΔX(M), ΔY(1)), (ΔX(M), ΔY(N));

[0024] S44. Generate a theoretical light spot array using the four-corner coordinates of the light spot array. The theoretical light spot array represents the light spot distribution during vertical irradiation. Calculate the horizontal and vertical spacings of the four-corner coordinates. Divide the horizontal and vertical spacings into M×N equal parts to generate M×N theoretical light spot array element coordinates.

[0025] Preferably, the one-dimensional results ΔX and ΔY are specifically:

[0026]

[0027] Where: H is the height of the image, W is the width of the image;

[0028] The horizontal spacing is as follows:

[0029]

[0030] The vertical spacing is as follows:

[0031]

[0032] The specific coordinates of the M×N theoretical spot array elements are:

[0033] l ij=(ΔY(1)+(i-1)·η, ΔX(1)+(j-1)·μ), i=1, 2…M, j=1, 2…N

[0034] Among them, ΔX(M) is the position of the Mth peak in , ΔY(1) is the position of the first peak in , and ΔY(N) is the position of the Nth peak in .

[0035] Preferably, step S5 specifically includes:

[0036] S51. In the vicinity of each element of the theoretical grid, search for an actual light spot at the Gaussian centroid of the candidate area, record the actual light spot found as recognized, and record the corresponding position of the unrecognized point;

[0037] S52. Solve the projective transformation matrix through four non-collinear points according to the plane transformation relationship between each element in the theoretical grid and the actual light spot;

[0038] S53. Calculate the corresponding position of the unidentified point using the projection transformation matrix and the theoretical grid element position, thereby calculating the pixel position of the unidentified point;

[0039] S54. Segment the M×N light spot array points into seed points, optimize the positioning twice, and establish eight neighborhoods; start growing from the brightest point in the neighborhood of the actual light spot, select directions from the eight neighborhood arrays, and judge the growth conditions. If the neighboring seed points of the seed point do not grow, judge whether the threshold condition is met. When all seed points cannot grow, stop segmenting the spot's precise contour and calculate the contour Gaussian centroid O. i ′;

[0040] S54. Using all candidate points O i ′ and theoretical point l ij 'Recalculate the projection transformation matrix H', and use the recalculated projection transformation matrix H' to transform O i ′ is iteratively updated, that is, H′ is used to calculate the corresponding position of the unrecognized point, and then the generated result is used to correct O′i, and H″ is recalculated until the output O′i reaches stability.

[0041] Preferably, the plane transformation relationship from each element in the theoretical grid to the actual light spot is:

[0042]

[0043]

[0044] The projection transformation matrix is:

[0045]

[0046] The corresponding position of the unidentified point γi for:

[0047]

[0048] The eight-neighborhood delta is:

[0049] a=[(-1 -1) (0 -1) (1 -1) (1 0) (1 1) (0 1) (-1 1)(-1 0)]

[0050] The threshold conditions are:

[0051]

[0052] Among them, O i is the Gaussian centroid coordinate of the candidate region, i = 1, 2, 3…2×M×N; l ij is the light spot in the theoretical grid neighborhood in row i and column j, l ij (x) is l ij The corresponding horizontal axis, l ij (y) is l ij The corresponding vertical coordinate; p ij is the light spot detected in the i-th row and j-th column of the theoretical grid neighborhood, p ij (x) is p ij The corresponding horizontal axis, p ij (y) is p ij The corresponding vertical coordinate; γ i is the missing light spot in the light spot array, γ i (x) is γ i The corresponding horizontal axis, γ i (y) is γ i The corresponding vertical coordinate.

[0053] A laser spot array detection system for structured light parameter calibration, based on the laser spot array detection method for structured light parameter calibration, includes a structured light camera, an image correction module, a filtering module, a channel extraction module, a region screening module, a theoretical spot array generation module, a projection transformation module, a seed point segmentation module and an iterative correction module;

[0054] A structured light camera, used to capture a full-color image to be detected containing an M×N array of laser spots on a laser calibration plane;

[0055] An image correction module is used to perform distortion correction processing on the image to be detected using the calibrated internal lens distortion coefficient of the structured light camera to obtain a corrected image;

[0056] A filtering module, used for performing image filtering on the corrected image to reduce image noise;

[0057] The channel extraction module is used to decompose the filtered image into RGB three-channel images, extract the red channel, and filter out the interference of other channel colors on spot detection;

[0058] The region screening module is used to perform full-image adaptive threshold segmentation on the red channel image, perform erosion and dilation operations on the segmented image, and perform region evaluation on each connected domain, sort by region area, and screen the Gaussian centroid coordinates and corresponding channels of the first 2×M×N candidate regions;

[0059] Theoretical spot array generation module is used to use the position relationship and energy distribution of candidate points to perform energy integration operation and calculate the peak position, locate the position of the spot array and extract the center position of the M×N spot array to generate an equidistantly distributed theoretical spot array;

[0060] The projection transformation module is used to calculate the position of the actual light spot using the position of the theoretical light spot array and the projection transformation algorithm, and to complete the missing light spot;

[0061] A seed point segmentation module is used to perform secondary positioning correction on the light spot array using a seed point segmentation algorithm;

[0062] The iterative correction module is used to iteratively perform projective transformation to correct the light spot deviation, and finally fit the straight line to obtain the sub-pixel light spot position.

[0063] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements a laser spot array detection method for structured light parameter calibration.

[0064] A processing terminal comprises a memory and a processor. The memory stores a computer program that can be run on the processor. When the processor executes the computer program, a laser spot array detection method for structured light parameter calibration is implemented.

[0065] It can be seen from the above technical solution that, compared with the prior art, the present invention discloses a laser spot array detection method and system for structured light parameter calibration, which can automatically locate the position of the M×N array spot captured by the camera based on the M×N array spot image captured by an uncalibrated structured light camera in the calibration state, without manual marking, thereby reducing the difficulty of structured light camera pipeline calibration; the shooting of the M×N array spot image may have spot-like noise and spot missing phenomena, thereby improving reliability and robustness; the M×N array spot positioning accuracy can reach the sub-pixel level, thereby improving the accuracy of structured light camera parameter calibration, which is of great significance for completing the automated calibration of the structured light endoscope system. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0067] Figure 1 A schematic diagram of a laser spot array detection method for structured light parameter calibration provided by the present invention;

[0068] Figure 2 A schematic diagram of various indicators for candidate point evaluation and screening provided by the present invention;

[0069] Figure 3 A schematic diagram of the detection site of the energy integration method provided by the present invention;

[0070] Figure 4 A schematic diagram of the X-axis integration result provided by the present invention;

[0071] Figure 5 A schematic diagram of the Y-axis integration result provided by the present invention;

[0072] Figure 6 A schematic diagram of the segmentation area provided by the present invention;

[0073] Figure 7 A schematic diagram of viewing angle transformation provided by the present invention;

[0074] Figure 8 A schematic diagram of the missing position coordinates calculated by the plane transformation provided by the present invention. DETAILED DESCRIPTION

[0075] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0076] In practical applications, the structured light camera is a combination of a monocular camera and a projection system, which can emit a laser spot array with a specific distribution pattern; the distribution pattern of the laser spot array is determined to be an M×N distribution pattern, and the laser spots are equidistantly distributed and the brightness between the spots is basically the same when irradiated vertically; the laser spot should have a certain grayscale difference with the background, and the brightness of the spot needs to be higher than the brightness of the working background, and can be segmented by an adaptive threshold segmentation algorithm; the calibration environment may have certain surface absorption and surface reflection phenomena, but it is necessary to ensure that there are no concave and convex surfaces or steps on the calibration plane; the surface absorption phenomenon exists but should not destroy the distribution pattern of the spot array, that is, there should not be a row or column that is completely absorbed so that it cannot be detected; the spot array image taken by the structured light camera of the present invention is segmented into multiple candidate areas through adaptive threshold detection, and the candidate areas are screened. The energy integration operation is performed on the screened area, the peak position is calculated, the position of the spot array is located, and the theoretical spot array is generated. The position of the actual spot is calculated from the theoretical spot using projection transformation to complete the missing spot. The seed point segmentation algorithm is used to perform secondary positioning correction on the light spot. According to the distribution pattern information of the light spot array, the light spot deviation is corrected by iterative projective transformation to obtain the sub-pixel light spot position.

[0077] Embodiment 1 discloses a laser spot array detection method for structured light parameter calibration, such as Figure 1 , including the following steps:

[0078] S1. Use a structured light camera to capture a full-color image I1 to be detected containing an M×N array of laser spots on a laser calibration plane, and use the internal lens distortion coefficient of the calibrated structured light camera to perform distortion correction on the image to be detected to obtain a corrected image I2 in RGB format, and perform image filtering on the corrected image to reduce image noise;

[0079] S2. decompose the filtered image into RGB three-channel images, extract the red channel, and filter out the interference of other channel colors on spot detection;

[0080] S3. Perform full-image adaptive threshold segmentation on the red channel image, that is, divide the entire image into patches, calculate an adaptive threshold for each path, segment the entire image, and use a 3×3 template to erode and dilate the segmented image to filter out the interference of small segmented areas on positioning accuracy, and for each connected domain Ω i Perform regional evaluation, sort by area and select the Gaussian centroid coordinates O of the first 2×M×N candidate regions i and corresponding channels;

[0081] S4. Using the position relationship and grayscale distribution of the candidate points, perform energy integration operation and calculate the peak position, locate the position of the spot array and extract the center position of the M×N spot array, and generate a theoretical spot array with equidistant distribution;

[0082] S5. Using the position of the theoretical light spot array, the actual light spot position is calculated by projective transformation, and the missing light spots are completed. The seed point segmentation algorithm is used to perform secondary positioning correction on the light spot array, and the projective transformation is iteratively performed to correct the light spot deviation. Finally, the straight line is fitted to obtain the sub-pixel light spot position.

[0083] In order to further implement the above technical solution, the image filtering process is specifically to continuously perform 5×5 Gaussian filtering, mean filtering, and 3×3 Gaussian filtering.

[0084] In this embodiment, the filter template is:

[0085]

[0086] Among them: gaus_kernel1 is the Gaussian kernel for the first filtering, gaus_kernel2 is the Gaussian kernel for the second filtering, and avr_kernel is the template for mean filtering.

[0087] In order to further implement the above technical solution, the method for regional evaluation of each connected domain is as follows:

[0088]

[0089] Among them: r means passing the evaluation system, b means the indicator exceeds the maximum threshold, s means the indicator is below the minimum threshold, is the connected domain Ω i area max 、area min The upper and lower bounds of the connected domain area are usually taken into account the actual spot size. max =1500, area min =50; is the connected domain Ω i The aspect ratio of the bounding rectangle, ratio max 、ratio min The upper and lower bounds of the aspect ratio of the circumscribed rectangle are usually taken into account considering the actual spot shape. max =1.5, ratio min =0.67; is the intensity of the red channel at the Gaussian centroid of the connected domain; R the is the minimum threshold of the intensity of the red channel at the Gaussian centroid of the spot, such as Figure 2 ;

[0090] After satisfying three evaluations, save the Gaussian centroid position O of the area i , sort by area and save the centroids of the first 2×M×N areas. If there are less than 2×M×N areas, save all of them.

[0091] In order to further implement the above technical solution, Figure 3-Figure 6 , step S4 specifically includes:

[0092] S41. Using the maximum area of ​​the candidate region S max , calculate the maximum radius R of the spot max , use the Gaussian centroid coordinates and the maximum spot radius to generate a template for the entire image, that is, for each Gaussian centroid coordinate O i , i = 1, 2, 3…2×M×N, a circular MASK segmentation image is generated with the maximum radius of the spot as the radius;

[0093] Maximum spot radius R max for:

[0094]

[0095] S42. Integrate the segmented image about the X and Y axes respectively to obtain two one-dimensional results ΔX and ΔY after integration;

[0096] S43. Due to the distribution characteristics and features of the light spot, the M×N light spot array should have MN peaks on ΔX and ΔY, and the distances between the MN peaks should not differ too much. The light spots are positioned using the peaks of the two integrated one-dimensional results, and the four corner coordinates of the light spot array are positioned using the positions of the first peak ΔX(1) in ΔX and the Mth peak ΔX(M) (as well as the first peak ΔY(1) in ΔY and the Nth peak ΔY(N) (ΔX(1), ΔY(1)), (ΔX(1), ΔY(N)), (ΔX(M), ΔY(1)), (ΔX(M), ΔY(N));

[0097] S44. Generate a theoretical light spot array L using the four-corner coordinates of the light spot array. The theoretical light spot array represents the light spot distribution during vertical irradiation. Calculate the horizontal and vertical spacings of the four-corner coordinates. The horizontal and vertical spacings are equally divided into M×N parts to generate the M×N theoretical light spot array element coordinates l ij .

[0098] In order to further implement the above technical solution, the one-dimensional results ΔX and ΔY are specifically:

[0099]

[0100] Where: H is the height of the image, W is the width of the image;

[0101] The horizontal spacing is as follows:

[0102]

[0103] The vertical spacing is as follows:

[0104]

[0105] The specific coordinates of the M×N theoretical spot array elements are:

[0106] l ij =(ΔY(1)+(i-1)·η,ΔX(1)+(j-1)·μ),i=1,2…M,j×1,2…N

[0107] Among them, ΔX(M) is the position of the Mth peak in , ΔY(1) is the position of the first peak in , and ΔY(N) is the position of the Nth peak in .

[0108] In order to further implement the above technical solution, step S5 specifically includes:

[0109] S51. In the theoretical grid, each element l ij In the vicinity, the Gaussian centroid O of the candidate region i Find the actual light spot p ij , for the actual light spot p found ij Record as identified, and record the corresponding position of the unidentified point γ i ;

[0110] S52. According to the plane transformation relationship from each element in the theoretical grid to the actual light spot, Figure 7 , solve the projective transformation matrix through 4 non-collinear points;

[0111] S53. Calculate the corresponding position of the unidentified point using the projection transformation matrix and the theoretical grid element position, thereby calculating the pixel position of the unidentified point, such as Figure 8 ;

[0112] S54. Segment the M×N light spot array points into seed points, optimize the positioning twice, and establish eight-neighborhood delta; from the actual light spot p ij Start growing from the brightest part of the neighboring area, select directions from the eight-neighbor delta array, and judge the growth conditions. If the neighboring seed point neighbor of the seed point seed does not grow, judge whether the threshold condition is met. When all seed points cannot grow, stop segmenting the spot's precise contour and calculate the contour Gaussian centroid O. i ′;

[0113] S54. Using all candidate points O′ iand theoretical point l′ ij Recalculate the projection transformation matrix H′, and use the recalculated projection transformation matrix H′ to transform O i ′ is iteratively updated, that is, H′ is used to calculate the corresponding position of the unrecognized point, and then the generated result is used to correct O′ i , recalculate H″ until output O i 'It reaches stability and basically no longer changes.

[0114] In order to further implement the above technical solution, since vertical shooting to actual angle shooting is an essential plane transformation relationship, each element l in the theoretical grid ij To the actual light spot p ij The plane transformation relationship is:

[0115]

[0116] Among them, the projection transformation matrix H can be set as

[0117]

[0118] Substituting H into the plane transformation equation, we get:

[0119]

[0120] Derived from the above formula:

[0121]

[0122] Given a projection transformation matrix H, multiplying its elements by the same number, the resulting homography a*H has the same effect as H, and a can be assumed to be 1 / h 22 , then H has only 8 free elements, and only 4 non-collinear points are needed to solve the projective transformation matrix H;

[0123] In practical applications, the projection transformation equation in the projection transformation algorithm includes 8 unknown quantities. A set of corresponding points can provide 2 sets of solution equations, so at least 4 sets of corresponding points are required to solve H. However, in order to ensure that the algorithm accuracy can reach the sub-pixel level, in actual situations, at least 4 sets of corresponding points are required. More than one group, that is, at least Each light spot can be identified by the adaptive threshold algorithm;

[0124] The corresponding position of the unidentified point γ i for:

[0125]

[0126] The eight-neighborhood delta is:

[0127] delat=[(-1 -1) (0 -1) (1 -1) (1 0) (1 1) (0 1) (-1 1) (-1 0)]

[0128] The threshold conditions are:

[0129]

[0130] Among them, O i is the Gaussian centroid coordinate of the candidate region, i = 1, 2, 3…2×M×N; l ij is the light spot in the theoretical grid neighborhood in row i and column j, l ij (x) is l ij The corresponding horizontal axis, l ij (y) is l ij The corresponding vertical coordinate; p ij is the light spot detected in the i-th row and j-th column of the theoretical grid neighborhood, p ij (x) is p ij The corresponding horizontal axis, p ij (y) is p ij The corresponding vertical coordinate; γ i is the missing light spot in the light spot array, γ i (x) is γ i The corresponding horizontal axis, γ i (y) is γ i The corresponding vertical coordinate.

[0131] The present invention provides data support for structured light camera parameter calibration by accurately positioning the laser spot array captured by the structured light camera, thereby improving the accuracy of structured light camera parameter calibration. There is no need to manually determine the spot position, and the missing spot can be calculated and generated, and iterative optimization is performed, thereby improving the reliability and robustness of the method.

[0132] Embodiment 2, a laser spot array detection system for structured light parameter calibration, based on a laser spot array detection method for structured light parameter calibration, includes a structured light camera, an image correction module, a filtering module, a channel extraction module, a region screening module, a theoretical spot array generation module, a projection transformation module, a seed point segmentation module and an iterative correction module;

[0133] A structured light camera, used to capture a full-color image to be detected containing an M×N array of laser spots on a laser calibration plane;

[0134] An image correction module is used to perform distortion correction processing on the image to be detected using the calibrated internal lens distortion coefficient of the structured light camera to obtain a corrected image;

[0135] A filtering module, used for performing image filtering on the corrected image to reduce image noise;

[0136] The channel extraction module is used to decompose the filtered image into RGB three-channel images, extract the red channel, and filter out the interference of other channel colors on spot detection;

[0137] The region screening module is used to perform full-image adaptive threshold segmentation on the red channel image, perform erosion and dilation operations on the segmented image, and perform region evaluation on each connected domain, sort by region area, and screen the Gaussian centroid coordinates and corresponding channels of the first 2×M×N candidate regions;

[0138] Theoretical spot array generation module is used to use the position relationship and energy distribution of candidate points to perform energy integration operation and calculate the peak position, locate the position of the spot array and extract the center position of the M×N spot array to generate an equidistantly distributed theoretical spot array;

[0139] The projection transformation module is used to calculate the position of the actual light spot using the position of the theoretical light spot array and the projection transformation algorithm, and to complete the missing light spot;

[0140] A seed point segmentation module is used to perform secondary positioning correction on the light spot array using a seed point segmentation algorithm;

[0141] The iterative correction module is used to iteratively perform projective transformation to correct the light spot deviation, and finally fit the straight line to obtain the sub-pixel light spot position.

[0142] Embodiment 3, a computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, a laser spot array detection method for structured light parameter calibration is implemented.

[0143] Embodiment 4, a processing terminal includes a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the processor executes the computer program, a laser spot array detection method for structured light parameter calibration is implemented.

[0144] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0145] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A laser spot array detection method for structured light parameter calibration, characterized in that: The following steps are involved: S1. Use a structured light camera to capture a full-color image to be detected containing an M×N array of laser spots on a laser calibration plane, and use the internal lens distortion coefficient of the calibrated structured light camera to perform distortion correction processing on the image to be detected to obtain a corrected image, and perform image filtering processing on the corrected image to reduce image noise; S2. decompose the filtered image into RGB three-channel images, extract the red channel, and filter out the interference of other channel colors on spot detection; S3. Perform full-image adaptive threshold segmentation on the red channel image, perform erosion and dilation operations on the segmented image, and perform regional evaluation on each connected domain, sort by area and screen the Gaussian centroid coordinates and corresponding channels of the first 2×M×N candidate regions; S4. Using the position relationship and grayscale distribution of the candidate points, perform energy integration operation and calculate the peak position, locate the position of the spot array and extract the center position of the M×N spot array, and generate a theoretical spot array with equidistant distribution; S5. Using the position of the theoretical light spot array, the actual light spot position is calculated by projective transformation, and the missing light spots are completed. The seed point segmentation algorithm is used to perform secondary positioning correction on the light spot array, and the projective transformation is iteratively performed to correct the light spot deviation. Finally, the straight line is fitted to obtain the sub-pixel light spot position.

2. The laser spot array detection method for structured light parameter calibration according to claim 1, characterized in that: The image filtering process specifically includes consecutively performing 5×5 Gaussian filtering, mean filtering, and 3×3 Gaussian filtering.

3. The laser spot array detection method for structured light parameter calibration according to claim 1, characterized in that: The method for regional evaluation of each connected domain is: Among them: r means passing the evaluation system, b means the indicator exceeds the maximum threshold, s means the indicator is below the minimum threshold, is the connected domain Ω i area max 、area min are the upper and lower bounds of the area of ​​the connected domain; is the connected domain Ω i The aspect ratio of the bounding rectangle, ratio max 、ratio min are the upper and lower bounds of the aspect ratio of the circumscribed rectangle; is the intensity of the red channel at the Gaussian centroid of the connected domain; R the It is the minimum threshold of the intensity of the red channel at the center of mass of the Gaussian spot.

4. The laser spot array detection method for structured light parameter calibration according to claim 1, characterized in that: Step S4 specifically includes: S41. Using the maximum area of ​​the candidate region, the maximum radius of the spot is calculated, and the Gaussian centroid coordinates and the maximum radius of the spot are used to generate a template for the entire image, that is, for each Gaussian centroid coordinate, a circular MASK segmentation image is generated with the maximum radius of the spot as the radius; S42. Integrate the segmented image about the X and Y axes respectively to obtain two one-dimensional results ΔX and ΔY after integration; S43. Use the peak values ​​of the two integrated one-dimensional results to locate the light spot, and use the four positions of the first peak position ΔX(1) and the Mth peak position ΔX(M) in ΔX and the first peak position ΔY(1) and the Nth peak position ΔY(N) in ΔY to locate the four corner coordinates of the light spot array (ΔX(1), ΔY(1)), (ΔX(1), ΔY(N)), (ΔX(M), ΔY(1)), (ΔX(M), ΔY(N)); S44. Generate a theoretical light spot array using the four-corner coordinates of the light spot array. The theoretical light spot array represents the light spot distribution during vertical irradiation. Calculate the horizontal and vertical spacings of the four-corner coordinates. Divide the horizontal and vertical spacings into M×N equal parts to generate M×N theoretical light spot array element coordinates.

5. The laser spot array detection method for structured light parameter calibration according to claim 3, characterized in that: The one-dimensional results ΔX and ΔY are specifically: Where: H is the height of the image, W is the width of the image; The horizontal spacing is as follows: The vertical spacing is as follows: The specific coordinates of the M×N theoretical spot array elements are: l ij =(ΔY(1)+(i-1)·η,ΔX(1)+(j-1)·μ),i=1,2…M,j=1,2…N Among them, ΔX(M) is the position of the Mth peak in, ΔY(1) is the position of the first peak in, and AY(N) is the position of the Nth peak in.

6. The laser spot array detection method for structured light parameter calibration according to claim 1, characterized in that: Step S5 specifically includes: S51. In the vicinity of each element of the theoretical grid, search for an actual light spot at the Gaussian centroid of the candidate area, record the actual light spot found as recognized, and record the corresponding position of the unrecognized point; S52. Solve the projective transformation matrix through four non-collinear points according to the plane transformation relationship between each element in the theoretical grid and the actual light spot; S53. Calculate the corresponding position of the unidentified point using the projection transformation matrix and the theoretical grid element position, thereby calculating the pixel position of the unidentified point; S54. Segment the M×N light spot array points into seed points, optimize the positioning twice, and establish eight neighborhoods; start growing from the brightest point in the neighborhood of the actual light spot, select directions from the eight neighborhood arrays, and judge the growth conditions. If the neighboring seed points of the seed point do not grow, judge whether the threshold condition is met. When all seed points cannot grow, stop segmenting the spot's precise contour and calculate the contour Gaussian centroid Q i ′; S54. Using all candidate points O i ′ and theoretical point l ij 'Recalculate the projection transformation matrix H', and use the recalculated projection transformation matrix H' to transform O i ′ is iteratively updated, that is, H′ is used to calculate the corresponding position of the unrecognized point, and then the generated result is used to correct O i ′, recalculate H" until output O i 'Achieve stability.

7. The laser spot array detection method for structured light parameter calibration according to claim 5, characterized in that: The plane transformation relationship from each element in the theoretical grid to the actual light spot is: The projection transformation matrix is: The corresponding position of the unidentified point γ i for: The eight-neighborhood delta is: delta=[(-1 -1) (0 -1) (1 -1) (1 0) (1 1) (0 1) (-1 1) (-1 0)] The threshold conditions are: Among them, O i is the Gaussian centroid coordinate of the candidate region, i = 1, 2, 3…2×M×N; l ij is the light spot in the theoretical grid neighborhood in row i and column j, l ij (x) is l ij The corresponding horizontal axis, l ij (y) is l ij The corresponding vertical coordinate; p ij is the light spot detected in the ith row and ith column of the theoretical grid neighborhood, p ij (x) is p ij The corresponding horizontal axis, p ij (y) is p ij The corresponding vertical coordinate; γ i is the missing light spot in the light spot array, γ i (x) is γ i The corresponding horizontal axis, γ i (y) is γ i The corresponding vertical coordinate.

8. A laser spot array detection system for structured light parameter calibration, characterized in that: A laser spot array detection method for structured light parameter calibration based on any one of claims 1 to 7, comprising a structured light camera, an image correction module, a filtering module, a channel extraction module, a region screening module, a theoretical spot array generation module, a projection transformation module, a seed point segmentation module and an iterative correction module; A structured light camera, used to capture a full-color image to be detected containing an M×N array of laser spots on a laser calibration plane; An image correction module is used to perform distortion correction processing on the image to be detected using the calibrated internal lens distortion coefficient of the structured light camera to obtain a corrected image; A filtering module, used for performing image filtering on the corrected image to reduce image noise; The channel extraction module is used to decompose the filtered image into RGB three-channel images, extract the red channel, and filter out the interference of other channel colors on spot detection; The region screening module is used to perform full-image adaptive threshold segmentation on the red channel image, perform erosion and dilation operations on the segmented image, and perform region evaluation on each connected domain, sort by region area, and screen the Gaussian centroid coordinates and corresponding channels of the first 2×M×N candidate regions; Theoretical spot array generation module is used to use the position relationship and grayscale distribution of candidate points to perform energy integration operation and calculate the peak position, locate the position of the spot array and extract the center position of the M×N spot array to generate an equidistantly distributed theoretical spot array; The projection transformation module is used to calculate the position of the actual light spot using the position of the theoretical light spot array and the projection transformation algorithm, and to complete the missing light spot; A seed point segmentation module is used to perform secondary positioning correction on the light spot array using a seed point segmentation algorithm; The iterative correction module is used to iteratively perform projective transformation to correct the light spot deviation, and finally fit the straight line to obtain the sub-pixel light spot position.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the laser spot array detection method for structured light parameter calibration described in any one of claims 1 to 7 is implemented.

10. A processing terminal, comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, characterized in that: When the processor executes the computer program, the laser spot array detection method for structured light parameter calibration as described in any one of claims 1 to 7 is implemented.

Images