A numbering method for a multi-line laser stripe under deformation and absence

Abstract

This invention discloses a numbering method for multi-line laser stripes under deformed or missing conditions. The method involves obtaining the depth values ​​of each point on the object under test in the camera coordinate system; setting the allowable height range for each pixel based on the depth values; capturing laser strip images with a camera; finding the connected components of the laser stripes and extracting the center line; calculating the spatial coordinates of the pixels on the center line; and determining the coordinate value z. j Whether it is within the height tolerance range corresponding to the pixel; count the z-values ​​that are within the height tolerance range. j The method counts the number of light stripes and calculates the ratio of this value to the total number of pixels. If the ratio is greater than the threshold, the equation of the light plane with the current number is stored in relation to the center line of the light stripe. If not, the equation of the light plane with the next number and the center line of the current light stripe are used to continue the judgment until all light stripes are traversed. This method still has good robustness when the laser stripe is deformed, missing, or occluded, and is suitable for the multi-line structured light plane calibration process of complex object surfaces.

Description

Technical Field

[0001] This invention relates to the field of optical plane calibration, specifically to a numbering method for multi-line laser strips that are deformed or missing. Background Technology

[0002] Optical 3D measurement technology, with its advantages of non-contact operation, high acquisition speed, and flexible equipment, is widely used in fields such as product reverse engineering, defect detection, robot navigation, augmented reality, and pose estimation. Among them, multi-line structured light measurement uses multi-line lasers and cameras to simultaneously project multiple laser lines onto the surface of the object under test. It can achieve both a large field of view and high-speed acquisition, and is suitable for measuring surfaces with strong textures. Therefore, multi-line laser sensors have been widely used in the field of structured light vision measurement.

[0003] In multi-line laser sensor applications, it is necessary to pre-calibrate the planar parameters of each laser plane sequentially to obtain the plane equation. Then, during 3D reconstruction, each laser stripe is extracted from the camera image, and the corresponding plane equation is looked up. Therefore, during 3D reconstruction, each laser stripe must correspond to the pre-calibrated plane equations. If confusion occurs, resulting in incorrect plane correspondence, it will cause significant 3D reconstruction errors and affect the accuracy of the measured structure.

[0004] Typical methods for matching multi-line laser stripes with a pre-calibrated light plane include: sorting method, regional parameter method, and binocular constraint method. Among them, the sorting method requires the camera to be able to clearly capture all the light stripes, and the laser stripes are arranged sequentially in a specific direction (e.g., from left to right). When the surface of the object being measured is complex, and there are occlusions or field-of-view problems, the acquired image will show problems such as missing light stripes, deformation, and occlusion, and the method will fail in this case.

[0005] The regional parameter method also requires the camera to clearly capture all light stripes. When the light stripes in the image are missing, the method fails. At the same time, this method needs to divide the field of view into a reference area and a measurement area. The light stripe number is estimated by the average offset of the pixel coordinates of each light stripe in the measurement area, which is not very robust.

[0006] The binocular constraint method requires an additional camera to acquire laser bar images and eliminate the singularity of each light bar number. When there is only one camera in the sensor, the method also needs to add a camera, which increases the system complexity and cost, making it difficult to implement. Summary of the Invention

[0007] To address the aforementioned technical problems, this invention proposes a numbering method for multi-line laser strips under deformed or missing conditions. Based on the morphology of the object under test, this method can accurately match the correct light plane for each laser strip in the image. It still exhibits good robustness when the laser strips are deformed, missing, or obstructed. It is suitable for the multi-line structured light plane calibration process on complex object surfaces and features low cost, high automation, and high accuracy.

[0008] Therefore, the technical solution of the present invention is as follows:

[0009] A numbering method for multi-line laser strips under deformed or missing conditions, wherein the multi-line laser sensor includes a multi-line laser projector and a camera, the multi-line laser projector being used to project multi-line laser strips onto the surface of the object to be tested, and the camera being used to acquire images of the laser strips;

[0010] Obtain the following information in advance:

[0011] Camera internal parameters;

[0012] The equations of each light plane in the camera coordinate system are generated, and the light plane equations are numbered sequentially according to a preset sorting method.

[0013] The depth values ​​of each point on the object under test within the detection area in the camera coordinate system are then used to set the depth values ​​for each pixel. j ,v j The corresponding allowable height range:

[0014] [(h(u j ,v j )-△h(u j ,v j )), (h(u j ,v j )+△h(u j ,v j ))]

[0015] Where, h(u j ,v j ) represents the j-th pixel (u j ,v j The depth value of the object to be measured at point (u) in the camera coordinate system; △h(u) j ,v j ) represents the j-th pixel (u j ,v j Preset height tolerance at () location;

[0016] During testing, the relative position of the multi-line laser sensor and the object under test is fixed. A batch of objects of the same model are placed in the testing area in sequence, and the pose deviation between each object under test is less than a preset value.

[0017] The multi-line laser projector projects multi-line laser stripes onto the object under test within the detection area, and the camera captures the laser stripe image;

[0018] Use the following steps to match the correct numbers to each laser bar in the laser bar image:

[0019] Step 1: Locate the connected components of the laser stripe in the laser stripe image, and extract the center line of the laser stripe in each connected component; number each center line of the laser stripe in the same sorting method as the optical plane equation, and when numbering, the value of the starting number of the laser stripe connected component should be consistent with the value of the starting number of the optical plane equation.

[0020] Perform step two using the center line of the light stripe with the starting number and the equation of the light plane with the starting number;

[0021] Step 2: Substitute the coordinates of each pixel on the center line of the light stripe into the following formula to calculate the coordinates of each pixel (u). j ,v j The corresponding spatial point coordinates (x) j ,y j ,z j ):

[0022]

[0023] Where f and d are the focal length and lateral pixel size obtained from camera calibration, respectively, (u'0, v'0) are the principal point coordinates, and A i x j +B i y j +C i z j +D=0 is the equation of the light plane with number i input this time; where the coordinate value z j Represents the coordinate value along the depth direction in the camera coordinate system;

[0024] Determine each coordinate value z separately j Is it at a pixel (u)? j ,v j The corresponding height allowable range [h(u)] j ,v j )-△h(u j ,v j ),h(u j ,v j )+△h(u j ,v j Within )]; and statistically analyze the coordinate values ​​z within the allowable height range. j The number of pixels is counted, and then the ratio between this value and the total number of pixels in the current connected component is calculated.

[0025] Step 3: If the ratio is greater than the threshold, store the current number of the light plane equation corresponding to the center line of the light stripe, and set the number of the center line of the light stripe to be equal to the number of the current light plane equation; then proceed to Step 4.

[0026] If the ratio is less than or equal to the threshold, then determine whether the current optical plane equation number is equal to the last number of the optical plane equation:

[0027] If so, mark the current light bar centerline as an invalid light bar and do not number it; proceed to step four;

[0028] If not, then continue with step two using the light plane equation corresponding to the next number value and the current light stripe centerline;

[0029] Step 4: Determine if there are any light bar centerlines that have not yet been executed in Step 2. If so, find the light bar centerline that is closest to the centerline of the starting number, and continue executing Step 2 using the found light bar centerline and the light plane equation of the current number. If not, the numbering ends.

[0030] Furthermore, the preset sorting methods include left to right, right to left, top to bottom, and bottom to top.

[0031] The present invention also discloses another numbering method for multi-line laser strips in the case of deformation or loss. The multi-line laser sensor includes a multi-line laser projector and a camera. The multi-line laser projector is used to project multi-line laser strips onto the surface of the object to be tested, and the camera is used to acquire laser strip images.

[0032] Its characteristic is that the following information is obtained in advance:

[0033] Camera internal parameters;

[0034] The equations of each light plane in the camera coordinate system are determined, and each light plane equation is numbered separately, with each equation having a unique number.

[0035] The depth values ​​of each point on the object under test within the detection area in the camera coordinate system are then used to set the depth values ​​for each pixel. j ,v j The corresponding allowable height range:

[0036] [(h(u j ,v j )-△h(u j ,v j )), (h(u j ,v j )+△h(u j ,v j ))]

[0037] Where, h(uj ,v j ) represents the j-th pixel (u j ,v j The depth value of the object to be measured at point (u) in the camera coordinate system; △h(u) j ,v j ) represents the j-th pixel (u j ,v j Preset height tolerance at () location;

[0038] During testing, the relative position of the multi-line laser sensor and the object under test is fixed. A batch of objects of the same model are placed in the testing area in sequence, and the pose deviation between each object under test is less than a preset value.

[0039] The multi-line laser projector projects multi-line laser stripes onto the object under test within the detection area, and the camera captures the laser stripe image;

[0040] Use the following steps to match the correct numbers to each laser bar in the laser bar image:

[0041] Step 1: Locate the connected components of the laser stripe in the laser stripe image, and extract the center line of the laser stripe in each connected component; number each center line of the laser stripe.

[0042] Choose any light stripe centerline and a random light plane equation and execute step two;

[0043] Step 2: Substitute the coordinates of each pixel on the center line of the light stripe into the following formula to calculate the coordinates of each pixel (u). j ,v j The corresponding spatial point coordinates (x) j ,y j ,z j ):

[0044]

[0045] Where f and d are the focal length and lateral pixel size obtained from camera calibration, respectively, (u'0, v'0) are the principal point coordinates, and A i x j +B i y j +C i z j +D=0 is the equation of the light plane with number i input this time; where the coordinate value z j Represents the coordinate value along the depth direction in the camera coordinate system;

[0046] Determine each coordinate value z separately j Is it at a pixel (u)? j ,v j The corresponding height allowable range [h(u)]j ,v j )-△h(u j ,v j ),h(u j ,v j )+△h(u j ,v j Within )]; and statistically analyze the coordinate values ​​z within the allowable height range. j The number of pixels is counted, and then the ratio between this value and the total number of pixels in the current connected component is calculated.

[0047] Step 3: If the ratio is greater than the threshold, store the current number of the light plane equation corresponding to the center line of the light stripe, and set the number of the center line of the light stripe to be equal to the number of the current light plane equation; then proceed to Step 4.

[0048] If the ratio is less than or equal to the threshold, continue to use other numbered light plane equations and the current light stripe centerline to continue executing step two until the ratio is greater than the threshold or all light plane equations have been traversed, and then execute step four.

[0049] Step 4: Determine if there are any light bar centerlines that have not yet been executed in Step 2. If so, select any light bar centerline and use that light bar centerline and any numbered light plane equation to continue executing Step 2. If not, the numbering ends.

[0050] Preferably, the preset tolerance △h(u) j ,v j The settings can be configured in two ways:

[0051] Method 1: Set based on experience values;

[0052] Method 2:

[0053]

[0054] Where L is the horizontal distance between the optical center of the laser and the optical center of the camera, and γ is the angle between the middle laser strip emitted by the laser and the optical axis of the camera.

[0055]

[0056] Where f, u0, and d are the focal length, lateral principal point coordinates, and lateral pixel size in the camera calibration results of step 1, respectively.

[0057] Preferably, the depth values ​​of each point on the object to be measured within the detection area in the camera coordinate system are obtained in advance, which can be achieved in the following two ways:

[0058] Method 1: Transform the modeling coordinate system to the camera coordinate system in the digital model of the object under test;

[0059] Method 2: Use a 3D measuring instrument to acquire 3D point cloud information on one of the objects to be measured, and calibrate the relationship between the coordinate system of the 3D measuring instrument and the camera coordinate system, and transform the 3D point cloud information to the camera coordinate system; the 3D measuring instrument includes a photogrammetry system and a 3D scanner.

[0060] Preferably, in step three, the threshold value is between 0.7 and 0.95.

[0061] Preferably, the camera's intrinsic parameters are calculated using Zhang's calibration or bundle adjustment methods.

[0062] The method of the present invention has the following characteristics:

[0063] It requires only one camera, without the need for other equipment, resulting in low cost and no increase in system complexity;

[0064] Based on the shape of the object under test, the height tolerance range is set according to the depth information corresponding to each pixel. Using the height tolerance range, the correct light plane is selected for each laser bar in the image, which is highly accurate and robust.

[0065] This method is applicable in situations including: when the camera cannot capture all light stripes within its field of view, when some light stripes are missing, when the light stripes are severely deformed, and when the light stripes are discontinuous (such as...). Figure 2 Compared to existing optical plane matching methods, this method has a wider range of applications. Therefore, even when the object to be measured has a complex shape, this method can still maintain its effectiveness and match the correct optical plane for the laser strip.

[0066] Instruction manual illustrations

[0067] Figure 1 To obtain △h(u) j ,v j A schematic diagram;

[0068] Figure 2 This is a two-dimensional image projected onto the surface of the object under test by a laser strip;

[0069] Figure 3 The image of the laser stripe captured by the camera;

[0070] Figure 4 This is a schematic diagram showing the laser strips after being numbered in step one;

[0071] Figure 5 This is a schematic diagram of the final laser bar numbering. Detailed Implementation

[0072] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0073] Example 1

[0074] In this embodiment, the numbering order of the light bar centerline and the light plane is the same (e.g., both from left to right). When matching the light bar centerline with the light plane (steps two to four), a number-by-number matching method is adopted. That is, step two is executed starting from the light bar centerline with the first number. Since the numbering order of the light bar centerline and the light plane is the same, the light bar centerline with the smaller number is more likely to match with the light plane with the smaller number, and the light bar centerline with the larger number is more likely to match with the light plane with the larger number. Based on this premise, the technical solution in this embodiment can improve the efficiency of numbering, especially suitable for situations with a large number of light bars.

[0075] The specific plan is as follows:

[0076] A method for numbering multi-line laser strips under conditions of laser strip deformation or loss, comprising a multi-line laser sensor including a multi-line laser projector and a camera, wherein the multi-line laser projector is used to project multi-line laser strips onto the surface of the object to be measured, and the camera is used to acquire laser strip images (e.g., ...). Figure 3 );

[0077] Obtain the following information in advance:

[0078] Camera internal parameters;

[0079] The equations of each light plane in the camera coordinate system are generated, and the light plane equations are numbered sequentially according to a preset sorting method.

[0080] The depth values ​​of each point on the object under test within the detection area in the camera coordinate system are then used to set the depth values ​​for each pixel. j ,v j The corresponding allowable height range:

[0081] [(h(u j ,v j )-△h(u j ,v j )), (h(u j ,v j )+△h(u j ,v j ))]

[0082] Where, h(u j ,v j ) represents the j-th pixel (u j ,v j The depth value of the object to be measured at point (u) in the camera coordinate system; △h(u) j ,v j ) represents the j-th pixel (u j ,v j Preset height tolerance at () location;

[0083] During testing, the relative position of the multi-line laser sensor and the object under test is fixed. A batch of objects of the same model are placed in the detection area in sequence, and the pose deviation between each object is less than a preset value; that is, the poses of multiple objects under test in the detection area are roughly the same, such as: all placed horizontally or all placed at an angle.

[0084] In practice, fixtures can be set up in the detection area, and the objects to be tested can be installed on the fixtures in sequence to ensure uniform posture; or, it is only necessary to rely on the positioning accuracy of the objects to be tested themselves, such as the objects to be tested being able to move and having small positioning accuracy deviations; or, the objects to be tested can be moved into the detection area by a robot, and the positioning deviation of the robot can be used to ensure uniform posture of each object to be tested and that the posture deviation is less than a preset value.

[0085] The multi-line laser projector projects multi-line laser stripes onto the object under test within the detection area, and the camera captures the laser stripe image;

[0086] Use the following steps to match the correct numbers to each laser bar in the laser bar image:

[0087] Step 1: Locate the connected components of the laser stripe in the laser stripe image, and extract the center line of the laser stripe within each connected component; number each center line of the laser stripe sequentially according to the same sorting method as the light plane equation (e.g., ...). Figure 4 When numbering, the starting number of the laser stripe connected region should be consistent with the starting number of the optical plane equation.

[0088] For example, if the order of the light plane equations is from left to right, starting from 3, then they are: light plane 3, light plane 4, and light plane 5. Then, the center lines of each light stripe are also numbered sequentially from left to right, starting from 3, and are: center line of light stripe 3, center line of light stripe 4, and center line of light stripe 5.

[0089] Perform step two using the center line of the light stripe with the starting number and the equation of the light plane with the starting number;

[0090] Step 2: Substitute the coordinates of each pixel on the center line of the light stripe into the following formula to calculate the coordinates of each pixel (u). j ,v j The corresponding spatial point coordinates (x) j ,y j ,z j ):

[0091]

[0092] Where f and d are the focal length and lateral pixel size obtained from camera calibration, respectively, (u'0, v'0) are the principal point coordinates, and A i x j +B i yj +C i z j +D=0 is the equation of the light plane with number i input this time; where the coordinate value z j Represents the coordinate value along the depth direction in the camera coordinate system;

[0093] Determine each coordinate value z separately j Is it at a pixel (u)? j ,v j The corresponding height allowable range [h(u)] j ,v j )-△h(u j ,v j ),h(u j ,v j )+△h(u j ,v j Within )]; and statistically analyze the coordinate values ​​z within the allowable height range. j The number of pixels is counted, and then the ratio between this value and the total number of pixels in the current connected component is calculated.

[0094] Step 3: If the ratio is greater than the threshold (the threshold value is between 0.7 and 0.95), store the current number of the light plane equation corresponding to the center line of the light stripe, and set the number of the center line of the light stripe to be equal to the number of the current light plane equation; then proceed to Step 4.

[0095] If the ratio is less than or equal to the threshold, then determine whether the current optical plane equation number is equal to the last number of the optical plane equation:

[0096] If so, mark the current light bar centerline as an invalid light bar and do not number it; proceed to step four;

[0097] If not, then continue with step two using the light plane equation corresponding to the next numbered value and the current light stripe centerline; (for example, if the light plane numbered 1 does not match the light stripe centerline numbered 1 (the ratio is less than the threshold), then use light plane number 2 and the light stripe centerline number 1 to execute step two. If they match, then change the number of the light stripe centerline number 1 from 1 to 2. If they still do not match, then use light plane number 3 and the light stripe centerline number 1 to execute step two...)

[0098] Step 4: Determine if there are any light bar centerlines that have not yet been executed in Step 2. If so, find the light bar centerline closest to the starting numbered light bar centerline, and continue executing Step 2 using the found light bar centerline and the current numbered light plane equation; (e.g.) Figure 4After step two is completed, the matching result is the center line of light plane 7 and the center line of light stripe 6. The center line of light stripe 6 is modified to the center line of light stripe 7. At this time, in step four, the center line of light stripe closest to the center line of the starting number light stripe among the center lines of light stripe that have not been executed in step two is the center line of light stripe 7. Step two is then executed again using the center line of light stripe 7 and the light plane 7.

[0099] If none, the numbering ends (e.g., ...). Figure 5 The middle has been revised Figure 4 The initial number of the error caused by the deformation of the light stripe was 6-24, and the corrected number is 7-24.

[0100] For example, the preset sorting methods that can be selected include left to right, right to left, top to bottom, and bottom to top.

[0101] More specifically, the preset tolerance △h(u) j ,v j The settings can be configured in two ways:

[0102] Method 1: Set based on empirical values; for example, the empirical value is the minimum difference in depth between two adjacent laser bars in the camera coordinate system.

[0103] Method 2:

[0104]

[0105] Among them, such as Figure 1 As shown, L is the horizontal distance between the optical center of the laser and the optical center of the camera, and γ is the angle between the middle laser strip emitted by the laser and the optical axis of the camera.

[0106]

[0107] Where f, u0, and d are the focal length, lateral principal point coordinates, and lateral pixel size in the camera calibration results of step 1, respectively.

[0108] The method for pre-acquiring the depth values ​​of each point on the object to be measured within the detection area in the camera coordinate system includes the following two approaches:

[0109] Method 1: Transform the modeling coordinate system to the camera coordinate system in the digital model of the object to be measured; when the object to be measured has a simple shape (such as in this embodiment). Figure 2 When measuring a stepped object, the distance between the object's coordinate system and the camera's coordinate system can be directly measured. The distance value can be used to directly correct the coordinate values ​​of each point on the surface being measured, and the corrected coordinate values ​​can be used as the depth values ​​in the camera's coordinate system.

[0110] Method 2: Use a 3D measuring instrument to acquire 3D point cloud information on one of the objects to be measured, and calibrate the relationship between the coordinate system of the 3D measuring instrument and the camera coordinate system, and transform the 3D point cloud information to the camera coordinate system; the 3D measuring instrument includes a photogrammetry system and a 3D scanner.

[0111] The prior information obtained by the above method is that the depth value of each point on the object under test in the camera coordinate system is an estimated value / predicted value. In the specific implementation process, through iteration (steps two to four) and probability statistics (determining whether the ratio is greater than the threshold in step three), the light plane with the highest probability of matching the light stripe is selected, thereby obtaining the correspondence between the light stripe and the light plane.

[0112] Specifically, the camera's intrinsic parameters are calculated using Zhang's calibration or bundle adjustment methods.

[0113] In this scheme, the shape of the multiple laser strips projected by the multi-line laser is not limited. Preferably, each laser strip is projected in approximately the same direction, such as approximately parallel multi-line laser strips or multi-line laser strips projected in both vertical and horizontal directions. When the projection directions of the multi-line laser strips are not uniform, such as grid-shaped multi-line laser strips, the horizontal laser strips can be numbered first, and then the vertical laser strips can be numbered.

[0114] Example 2

[0115] The difference between this embodiment and Embodiment 1 is that the numbering order of the light stripe centerline and the light plane can be randomly distributed. For example, the numbers of the three light planes from left to right are 3, 1, and 5, and the initial numbering of the light stripe centerline is also random, such as 4, 2, and 6 from left to right. When matching the light stripe centerline with the light plane (steps two to four), a random combination method is used. Finally, it is only necessary to find out the light plane that matches the light stripe centerline and then assign the number of the light plane to the number of the light stripe centerline.

[0116] Example 1 is more suitable for situations with a large number of light stripes, while this example is more suitable for situations with a small number of light stripes (such as when the number of light stripes is less than 8).

[0117] The specific technical solution is as follows:

[0118] A method for numbering multi-line laser stripes under deformed and missing conditions, wherein the multi-line laser sensor includes a multi-line laser projector and a camera, the multi-line laser projector is used to project multi-line laser stripes onto the surface of the object to be tested, and the camera is used to acquire laser stripe images;

[0119] Obtain the following information in advance:

[0120] Camera internal parameters;

[0121] The equations of each light plane in the camera coordinate system are determined, and each light plane equation is numbered separately, with each equation having a unique number.

[0122] The depth values ​​of each point on the object under test within the detection area in the camera coordinate system are then used to set the depth values ​​for each pixel. j ,v j The corresponding allowable height range:

[0123] [(h(u j ,v j )-△h(u j ,v j )), (h(u j ,v j )+△h(u j ,v j ))]

[0124] Where, h(u j ,v j ) represents the j-th pixel (u j ,v j The depth value of the object to be measured at point (u) in the camera coordinate system; △h(u) j ,v j ) represents the j-th pixel (u j ,v j Preset height tolerance at () location;

[0125] During testing, the relative position of the multi-line laser sensor and the object under test is fixed. A batch of objects of the same model are placed in the testing area in sequence, and the pose deviation between each object under test is less than a preset value.

[0126] The multi-line laser projector projects multi-line laser stripes onto the object under test within the detection area, and the camera captures the laser stripe image;

[0127] Use the following steps to match the correct numbers to each laser bar in the laser bar image:

[0128] Step 1: Locate the connected components of the laser stripe in the laser stripe image, and extract the center line of the laser stripe in each connected component; number each center line of the laser stripe.

[0129] Choose any light stripe centerline and a random light plane equation and execute step two;

[0130] Step 2: Substitute the coordinates of each pixel on the center line of the light stripe into the following formula to calculate the coordinates of each pixel (u). j ,v j The corresponding spatial point coordinates (x) j ,y j ,z j ):

[0131]

[0132] Where f and d are the focal length and lateral pixel size obtained from camera calibration, respectively, (u'0, v'0) are the principal point coordinates, and A i x j +B i y j +C i z j +D=0 is the equation of the light plane with number i input this time; where the coordinate value z j Represents the coordinate value along the depth direction in the camera coordinate system;

[0133] Determine each coordinate value z separately j Is it at a pixel (u)? j ,v j The corresponding height allowable range [h(u)] j ,v j )-△h(u j ,v j ),h(u j ,v j )+△h(u j ,v j Within )]; and statistically analyze the coordinate values ​​z within the allowable height range. j The number of pixels is counted, and then the ratio between this value and the total number of pixels in the current connected component is calculated.

[0134] Step 3: If the ratio is greater than the threshold, store the current number of the light plane equation corresponding to the center line of the light stripe, and set the number of the center line of the light stripe to be equal to the number of the current light plane equation; then proceed to Step 4.

[0135] If the ratio is less than or equal to the threshold (e.g., 0.8), then continue to use other numbered light plane equations and the current light stripe centerline to continue executing step two until the ratio is greater than the threshold or all light plane equations have been traversed, and then execute step four.

[0136] Step 4: Determine if there are any light bar centerlines that have not yet been executed in Step 2. If so, select any light bar centerline and use that light bar centerline and any numbered light plane equation to continue executing Step 2. If not, the numbering ends.

[0137] More specifically, the preset tolerance △h(u) j ,v j The settings can be configured in two ways:

[0138] Method 1: Set based on empirical values; for example, the empirical value is the minimum difference in depth between two adjacent laser bars in the camera coordinate system.

[0139] Method 2:

[0140]

[0141] Among them, such as Figure 1 As shown, L is the horizontal distance between the optical center of the laser and the optical center of the camera, and γ is the angle between the middle laser strip emitted by the laser and the optical axis of the camera.

[0142]

[0143] Where f, u0, and d are the focal length, lateral principal point coordinates, and lateral pixel size in the camera calibration results of step 1, respectively.

[0144] The method for pre-acquiring the depth values ​​of each point on the object to be measured within the detection area in the camera coordinate system includes the following two approaches:

[0145] Method 1: Transform the modeling coordinate system to the camera coordinate system in the digital model of the object under test;

[0146] Method 2: Use a 3D measuring instrument to acquire 3D point cloud information on one of the objects to be measured, and calibrate the relationship between the coordinate system of the 3D measuring instrument and the camera coordinate system, and transform the 3D point cloud information to the camera coordinate system; the 3D measuring instrument includes a photogrammetry system and a 3D scanner.

[0147] Specifically, the camera's intrinsic parameters are calculated using Zhang's calibration or bundle adjustment methods.

[0148] The foregoing description of specific exemplary embodiments of the present invention is for illustrative and descriptive purposes. It is not intended to be exhaustive, nor to limit the invention to the precise forms disclosed; obviously, many changes and variations are possible in accordance with the foregoing teachings. The exemplary embodiments were chosen and described to explain the specific principles of the invention and its practical application, thereby enabling others skilled in the art to implement and utilize various exemplary embodiments of the invention, as well as their different alternatives and modifications. The scope of the invention is intended to be defined by the appended claims and their equivalents.

Claims

1. A numbering method for multi-line laser strips under deformed or missing conditions, wherein the multi-line laser sensor includes a multi-line laser projector and a camera, the multi-line laser projector being used to project multi-line laser strips onto the surface of the object to be tested, and the camera being used to acquire images of the laser strips; Its features are, Obtain the following information in advance: Camera internal parameters; The equations of each light plane in the camera coordinate system are generated, and the light plane equations are numbered sequentially according to a preset sorting method. The depth values ​​of each point on the object under test within the detection area in the camera coordinate system are then used to set the depth values ​​for each pixel. j ,v j The corresponding allowable height range: [(h(u j ,v j )-△h(u j ,v j )),(h(u j ,v j )+△h(u j ,v j ))] Where, h(u j ,v j ) represents the j-th pixel (u j ,v j The depth value of the object to be measured at point (u) in the camera coordinate system; △h(u) j ,v j ) represents the j-th pixel (u j ,v j Preset height tolerance at () location; During testing, the relative position of the multi-line laser sensor and the object under test is fixed. A batch of objects of the same model are placed in the testing area in sequence, and the pose deviation between each object under test is less than a preset value. The multi-line laser projector projects multi-line laser stripes onto the object under test within the detection area, and the camera captures the laser stripe image; Use the following steps to match the correct numbers to each laser bar in the laser bar image: Step 1: Locate the connected components of the laser stripe in the laser stripe image, and extract the center line of the laser stripe in each connected component; number each center line of the laser stripe in the same sorting method as the optical plane equation, and when numbering, the value of the starting number of the laser stripe connected component should be consistent with the value of the starting number of the optical plane equation. Perform step two using the center line of the light stripe with the starting number and the equation of the light plane with the starting number; Step 2: Substitute the coordinates of each pixel on the center line of the light stripe into the following formula to calculate the coordinates of each pixel (u). j ,v j The corresponding spatial point coordinates (x) j ,y j ,z j ): Where f and d are the focal length and lateral pixel size obtained from camera calibration, respectively, (u'0, v'0) are the principal point coordinates, and A i x j +B i y j +C i z j +D=0 is the equation of the light plane with number i input this time; where the coordinate value z j Represents the coordinate value along the depth direction in the camera coordinate system; Determine each coordinate value z separately j Is it at a pixel (u)? j ,v j The corresponding height allowable range [h(u)] j ,v j )-△h(u j ,v j ),h(u j ,v j )+△h(u j ,v j Within )]; and statistically analyze the coordinate values ​​z within the allowable height range. j The number of pixels is counted, and then the ratio between that value and the total number of pixels in the current connected component is calculated. Step 3: If the ratio is greater than the threshold, store the current number of the light plane equation corresponding to the center line of the light stripe, and set the number of the center line of the light stripe to be equal to the number of the current light plane equation; then proceed to Step 4. If the ratio is less than or equal to the threshold, then determine whether the current optical plane equation number is equal to the last number of the optical plane equation: If so, mark the current light bar centerline as an invalid light bar and do not number it; proceed to step four; If not, then continue with step two using the light plane equation corresponding to the next number value and the current light stripe centerline; Step 4: Determine if there are any light bar centerlines that have not yet been executed in Step 2. If so, find the light bar centerline that is closest to the centerline of the starting number, and continue executing Step 2 using the found light bar centerline and the light plane equation of the current number. If not, the numbering ends.

2. The numbering method as described in claim 1, characterized in that: The preset sorting methods include left to right, right to left, top to bottom, and bottom to top.

3. A numbering method for multi-line laser strips under deformed or missing conditions, wherein the multi-line laser sensor includes a multi-line laser projector and a camera, the multi-line laser projector being used to project multi-line laser strips onto the surface of the object to be tested, and the camera being used to acquire images of the laser strips; Its features are, Obtain the following information in advance: Camera internal parameters; The equations of each light plane in the camera coordinate system are determined, and each light plane equation is numbered separately, with each equation having a unique number. The depth values ​​of each point on the object under test within the detection area in the camera coordinate system are then used to set the depth values ​​for each pixel. j ,v j The corresponding allowable height range: [(h(u j ,v j )-△h(u j ,v j )),(h(u j ,v j )+△h(u j ,v j ))] Where, h(u j ,v j ) represents the j-th pixel (u j ,v j The depth value of the object to be measured at point (u) in the camera coordinate system; △h(u) j ,v j ) represents the j-th pixel (u j ,v j Preset height tolerance at () location; During testing, the relative position of the multi-line laser sensor and the object under test is fixed. A batch of objects of the same model are placed in the testing area in sequence, and the pose deviation between each object under test is less than a preset value. The multi-line laser projector projects multi-line laser stripes onto the object under test within the detection area, and the camera captures the laser stripe image; Use the following steps to match the correct numbers to each laser bar in the laser bar image: Step 1: Locate the connected components of the laser stripe in the laser stripe image, and extract the center line of the laser stripe in each connected component; number each center line of the laser stripe. Choose any light stripe centerline and a random light plane equation and execute step two; Step 2: Substitute the coordinates of each pixel on the center line of the light stripe into the following formula to calculate the coordinates of each pixel (u). j ,v j The corresponding spatial point coordinates (x) j ,y j ,z j ): Where f and d are the focal length and lateral pixel size obtained from camera calibration, respectively, (u'0, v'0) are the principal point coordinates, and A i x j +B i y j +C i z j +D=0 is the equation of the light plane with number i input this time; where the coordinate value z j Represents the coordinate value along the depth direction in the camera coordinate system; Determine each coordinate value z separately j Is it at a pixel (u)? j ,v j The corresponding height allowable range [h(u)] j ,v j )-△h(u j ,v j ),h(u j ,v j )+△h(u j ,v j Within )]; and statistically analyze the coordinate values ​​z within the allowable height range. j The number of pixels is counted, and then the ratio between that value and the total number of pixels in the current connected component is calculated. Step 3: If the ratio is greater than the threshold, store the current number of the light plane equation corresponding to the center line of the light stripe, and set the number of the center line of the light stripe to be equal to the number of the current light plane equation; then proceed to Step 4. If the ratio is less than or equal to the threshold, continue to use other numbered light plane equations and the current light stripe centerline to continue executing step two until the ratio is greater than the threshold or all light plane equations have been traversed, and then execute step four. Step 4: Determine if there are any light bar centerlines that have not yet been executed in Step 2. If so, select any light bar centerline and use that light bar centerline and any numbered light plane equation to continue executing Step 2. If not, the numbering ends.

4. The numbering method as described in claim 1 or 3, characterized in that: The preset tolerance △h(u j ,v j The settings can be configured in two ways: Method 1: Set based on experience values; Method 2: Where L is the horizontal distance between the optical center of the laser and the optical center of the camera, and γ is the angle between the middle laser strip emitted by the laser and the optical axis of the camera. Where f, u0, and d are the focal length, lateral principal point coordinates, and lateral pixel size in the camera calibration results of step 1, respectively.

5. The numbering method as described in claim 1 or 3, characterized in that: Pre-acquiring the depth values ​​of each point on the object under test within the detection area in the camera coordinate system can be achieved in the following two ways: Method 1: Transform the modeling coordinate system to the camera coordinate system in the digital model of the object under test; Method 2: Use a 3D measuring instrument to acquire 3D point cloud information on one of the objects to be measured, and calibrate the relationship between the coordinate system of the 3D measuring instrument and the camera coordinate system, and transform the 3D point cloud information to the camera coordinate system; the 3D measuring instrument includes a photogrammetry system and a 3D scanner.

6. The numbering method as described in claim 1 or 3, characterized in that: In step three, the threshold value is set between 0.7 and 0.

95.

7. The numbering method as described in claim 1 or 3, characterized in that: The camera's intrinsic parameters are calculated using Zhang's calibration or bundle adjustment methods.

Images