Complex surface structured light contour extraction method, electronic equipment and storage medium

Abstract

The invention relates to a complex surface structured light contour extraction method, electronic equipment and a storage medium, and the method comprises the steps: obtaining a line structured light original image of a complex surface, and extracting pixel points higher than a preset threshold value in each column or each row as contour points in a column or row scanning mode; for a column or row with only one contour point, adding the contour points of the column or row into a reference pool; for the column or row without the contour point, gradually reducing the threshold value for scanning until at least one contour point exists in the current column or row, or the threshold value is lower than a preset minimum threshold value; calculating the distance between adjacent contour points in the reference pool to obtain a structured light contour basic form entropy value; for a column or row with a plurality of contour points, the contour point with the minimum entropy increment of the morphological entropy in the column or row is reserved; and for the columns or rows without the contour points, ensuring that each column or each row has at least one contour point through interpolation, and obtaining the complex surface structured light contour.

Description

Technical Field

[0001] The present invention relates to the technical field of structured light detection, and in particular to a complex surface structured light contour extraction method, electronic equipment, and storage medium. Background Art

[0002] The line structured light 3D detection method based on the triangulation principle has been successfully applied in many fields, with the advantages of simple hardware, high measurement accuracy, and fast detection speed. The height change of the surface of the measured object is reflected in the structured light profile. The complete and accurate extraction of the structured light profile is the basis for obtaining height information. However, in scenes such as uneven reflectivity of the material surface, fluctuating ambient light, and complex surface profile, the structured light profile extraction is difficult or erroneous. The main difficulties are:

[0003] (1) Differences in material reflectivity lead to different brightness in different areas of the structured light profile, and there is a lack of standards for the selection and optimization of the extraction threshold;

[0004] (2) Complex structures can easily lead to multiple reflections of structured light, causing the position and height information of some stripes to not satisfy the triangulation principle, and the resulting measurement deviation is difficult to detect.

[0005] In recent years, many studies have explored solutions from different angles. For example, the invention patent application with Chinese patent application number 202410171355.0, "A Linear Structured Light Contour Measurement Method for High-speed Rails", builds a structured light binocular measurement system and uses two imaging systems to improve the integrity of contour measurement data; the invention patent application with Chinese patent application number 202410670377.1, "Scanning Structured Light Center Contour Extraction for Objects with Uneven Reflective Light", uses auxiliary LED lighting and matching cameras to determine the structured light contour extraction threshold based on the reflection state of the measured surface to other bands of light sources. These methods help to improve the structured light contour extraction effect, but the hardware system needs to be added, which makes the measurement device larger and more expensive.

[0006] There are also studies that attempt to improve the anti-interference ability of structured light contour extraction from an algorithmic perspective. For example, the 2024 master's thesis of Changchun University of Technology, "Research on Weld Forming Quality Detection Method Based on Structured Light Vision", proposes a least squares polynomial segmented fitting extraction method to solve the problem of pseudo-corner interference of structured light contours. The fine extraction of feature points is achieved by comparing the difference between the intersection and the vertical axis distance of the fitting curve. However, the curve fitting method is not applicable enough in complex contour scenes. The paper "Strong Noise Weld Recognition Method Based on Improved Active Contour Model" (Hot Working Technology, Vol. 54, No. 3) optimizes the energy function of the structured light contour model according to the grayscale distribution characteristics of the noise image to improve the stability of the convergence result. However, this method mainly considers arc light interference and lacks the ability to suppress the interference of multiple reflections of structured light caused by complex surfaces.

[0007] In summary, there is currently a lack of a structured light contour extraction method for non-uniform materials and complex structure surfaces, so as to improve the measurement accuracy, completeness and scope of application of line structured light three-dimensional measurement technology. Summary of the invention

[0008] The purpose of the present invention is to overcome the defects of the above-mentioned prior art and provide a complex surface structured light contour extraction method, electronic device, and storage medium to solve or partially solve the problem of contour missing and pseudo contour having poor adaptability to complex surface structured light contour extraction.

[0009] The purpose of the present invention can be achieved by the following technical solutions:

[0010] One aspect of the present invention provides a complex surface structured light contour extraction method, comprising the following steps:

[0011] Obtain the original line structured light image of the complex surface, and extract the pixel points in each column or row that are higher than a preset threshold as contour points in a column or row scanning manner;

[0012] For a column or row with only one contour point, the contour point of the column or row is added to the reference pool;

[0013] For columns or rows without contour points, the threshold is gradually lowered to scan until at least one contour point exists in the current column or row, or the threshold is lower than the preset minimum threshold;

[0014] Calculating the distances between adjacent contour points in the reference pool to obtain a basic morphological entropy value of the structured light contour;

[0015] For a column or row with multiple contour points, retain the contour point with the smallest entropy increment of the morphological entropy in the column or row;

[0016] For columns or rows without contour points, interpolation is performed to ensure that there is at least one contour point on each column or each row, thereby obtaining the complex surface structured light contour.

[0017] As a preferred technical solution, when gradually lowering the threshold for scanning, the following steps are also included:

[0018] If there is only one contour point in a column or row after lowering the threshold scan, the contour point is added to the candidate pool;

[0019] The multi-neighborhood directional gradients of the contour points in the candidate pool are calculated, and contour points with similar directional gradients are screened out.

[0020] As a preferred technical solution, calculating the multi-neighborhood directional gradients of the contour points in the candidate pool, and filtering out the contour points with similar directional gradients includes the following steps:

[0021] Based on the color of the line structured light, a color difference map corresponding to the original image of the line structured light is obtained;

[0022] For each contour point in the candidate pool, based on the color difference map, the gradient within the N neighborhood range is calculated, and the gradient value is searched according to the interval interval. When there is only one gradient value that is not in the reference interval and the other gradient values ​​are all in the reference interval, the distances between the gradient value point outside the reference interval and the maximum gradient value point and the minimum gradient value point in the reference interval are calculated respectively, and it is determined whether the distances are greater than the average spacing of the preset multiples in the reference interval. If so, the current contour point is retained. If not, the neighborhood range is expanded and the search is performed again. When the neighborhood range exceeds the preset value, the contour point is deleted.

[0023] If the gradients of the contour points in each direction are close, they will be removed. If the gradient of the contour points in any direction is much larger than the gradients in other directions, they will be retained.

[0024] As a preferred technical solution, the interval is a preset value, or is updated according to a preset interval.

[0025] As a preferred technical solution, the basic morphological entropy value is the cumulative sum of the distances between two adjacent contour points in the reference pool.

[0026] As a preferred technical solution, the process of retaining the contour points with the smallest entropy increment of the morphological entropy in the column or row includes:

[0027] Calculate the sum of the distances between each contour point in the column or row and the adjacent contour points on both sides of the reference pool when adding them to the reference pool, as the morphological entropy value, and subtract it from the basic morphological entropy value to obtain the entropy value of the morphological entropy.

[0028] As a preferred technical solution, the interpolation is cubic spline interpolation.

[0029] As a preferred technical solution, the scanning is line-by-line scanning or scanning at preset intervals.

[0030] Another aspect of the present invention provides an electronic device, comprising: one or more processors and a memory, wherein the memory stores one or more programs, and the one or more programs include instructions for executing the aforementioned complex surface structured light contour extraction method.

[0031] Another aspect of the present invention provides a computer-readable storage medium, comprising one or more programs for execution by one or more processors of an electronic device, wherein the one or more programs include instructions for executing the aforementioned complex surface structured light contour extraction method.

[0032] Compared with the prior art, the present invention has at least one of the following beneficial effects:

[0033] (1) Effectively deal with possible pseudo contours: The present invention calculates the basic morphological entropy value of the structured light contour and retains the contour point with the smallest entropy value in the column or row for a column or row with multiple contour points. This can effectively eliminate the pseudo contour caused by the reflection of the structured light itself, thereby ensuring the accuracy of the final extracted structured light contour.

[0034] (2) Effectively deal with possible contour missing: The present invention reduces the threshold scanning and uses interpolation to ensure that there is at least one contour point on each column or each row to obtain a complex surface structured light contour. It can have good adaptability to local contour missing and interference, especially dark area contour missing caused by differences in material reflectivity.

[0035] (3) No need to add hardware: This method does not require any additional hardware in the line structured light measurement system. Through the design of the structured light contour extraction strategy, both the integrity and accuracy of the contour are taken into account.

[0036] (4) Suitable for structured light contour extraction of complex structures: According to the characteristics of line structured light images, the present invention designs a grouping processing mode by analyzing the idea of ​​artificially identifying line structured light contours. The mode has good adaptability, can meet the height measurement of material surfaces with different degrees of non-uniformity, and has good anti-interference ability for structured light reflection interference caused by complex structures. It has fast calculation speed and high algorithm stability. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 It is a schematic diagram of the process of extracting the complex surface structured light contour in the embodiment;

[0038] Figure 2 An original image of an exemplary line structured light in the embodiment;

[0039] Figure 3This is a schematic diagram of scanning according to a preset threshold value in the embodiment;

[0040] Figure 4 This is a schematic diagram of the scanning after the threshold value is lowered in the embodiment;

[0041] Figure 5 This is a schematic diagram of retaining the contour point with the smallest entropy increment of the morphological entropy in the column or row in the embodiment;

[0042] Figure 6 This is the complex surface structure light profile finally obtained in the embodiment. DETAILED DESCRIPTION

[0043] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0044] Example 1

[0045] In response to the problems existing in the aforementioned prior art, this embodiment provides a method for extracting structured light contours from complex surfaces. The basic idea is: scan the image vertically, calculate the center of the line structured light contour column by column or at specified intervals, and if there is only one contour point with brightness exceeding a threshold in a column, push the point into the reference pool; if there is no contour point with brightness exceeding the threshold in a column, gradually lower the threshold to find contour points, and push these contour points into the candidate group; when there are multiple contour points in a column, push all contour points into the competition group. After determining the grouping of contour points, the distance between adjacent points is calculated based on the reference pool, and the cumulative distance of all points in the reference pool is obtained as the basic entropy value; for the scanning vertical lines where candidate groups or competitive groups appear, the contour points are added to the reference pool respectively, and the distance between adjacent points after adding is calculated to obtain the entropy value increment after adding the points; for the scanning vertical lines with only one newly added point, if the entropy increment is less than the mode distance of the reference pool, the point is accepted as the structured light contour point; for the scanning vertical lines with multiple contour points, the contour point with the smallest entropy value increment is selected; finally, for the scanning vertical lines where contour points cannot be found even if the threshold is lowered, cubic spline interpolation is used to repair them, and finally a continuous and complete line structured light contour line is obtained.

[0046] See also Figure 1 , the method comprises the following steps:

[0047] Step S1, obtaining the original line structured light image of the complex surface and scanning the contour points larger than the preset ones by columns / rows to construct a reference pool.

[0048] Scan the image column by column or at fixed intervals, and filter out pixels with grayscale and gradient higher than the threshold as contour points. If there is only one contour point that meets the conditions in a column, the contour point is pushed into the "reference pool" as the main contour of the line structured light.

[0049] In practice, according to the direction of the line structured light in the image, the image can be rotated, cropped, and the like in advance, so that the line structured light is in the image along the horizontal or vertical direction, and then scanned in a column or row manner.

[0050] Step S2: For columns / rows without contour points, a search is performed by gradually lowering the threshold to construct a candidate pool.

[0051] For columns or rows where no contour points are found, the threshold is gradually lowered for searching until all contour points are found or the threshold is lowered to the set value. The contour points found after the threshold is lowered are pushed into the "candidate pool"; if multiple contour points appear in a column at the same time after the threshold is lowered, they are all pushed into the "candidate pool"; the scanned columns where contour points have been found do not participate in the threshold lowering search.

[0052] Step S3, for the candidate pool, by calculating the eight-neighborhood directional gradients, screen and retain contour points with significantly higher gradients in a certain direction.

[0053] The eight-neighborhood directional gradients are calculated for the contour points in the candidate pool. If the gradients in each direction are close, regardless of the gradient level, it will be judged as a noise point and deleted; if there is a direction whose gradient is significantly higher than other directions, it will be judged as a contour point.

[0054] Specifically, when calculating the eight-neighborhood direction gradient in step S3, the image is first converted into a color difference map matching the laser color, and then the grayscale gradient in the eight-neighborhood direction is calculated according to the color difference map. For example, when the laser line is blue, the grayscale gradient is calculated using a blue-red (red is the inverse color of the laser line) color difference map, which has a certain inhibitory effect on suppressing white or black noise caused by local reflection.

[0055] Specifically, the process of determining in step S3 that the gradients in various directions are close and the gradient in one direction is significantly higher than that in other directions is as follows:

[0056] First, set the gradient calculation step size, such as calculating the 8-neighborhood gradient at a length of 3 pixels; then calculate the mode range of the gradient value, for example, search at an interval of ±15. If there are 7 values ​​in a reference interval and only one value is not in the reference interval, calculate the distance from the value not in the reference interval to the maximum and minimum values ​​in the reference interval. If both distances exceed 1.5 times the average spacing of the points in the range, it is considered that the directional gradient of the value is significantly higher than that of other directions, and retain the contour point in the candidate pool; if it is not found in this round, expand the gradient calculation range, for example, perform the same calculation at a length of 6 pixels; when the search range exceeds the set value, it is considered that the gradients in each direction are close, and delete the contour point in the candidate pool.

[0057] The interval setting range may be 5-30, or an iterative method may be used, for example, changing from 5 to 30 at intervals of 5, and searching once. In this way, more candidate points can be searched in situations where there are many missing points to improve the extraction effect.

[0058] Step S4, for a scan column / row with multiple contour points, construct a competition group by column / row.

[0059] For scan columns with multiple contour points, the contour points are grouped into "competition groups" by column, and only one contour point in each column is allowed to win in the end.

[0060] Step S5: for the competitive group, select and retain the contour point with the smallest entropy increment.

[0061] Taking the contour points in the "reference pool" as the object, calculate the distance between adjacent points and accumulate them, and use the accumulated distance as the basic morphological entropy value of the trunk contour; for the scan columns with competitive groups, add the contour points to the trunk contour respectively, calculate the distance between the added points and the adjacent trunk contour points on both sides, use the sum of the distances on both sides as the morphological entropy increment, and select the contour point with the smallest entropy increase. Only one point is selected each time; the selected contour point does not participate in the subsequent entropy increase calculation.

[0062] Step S6: For columns / rows without contour points, cubic spline sampling is used to perform patching.

[0063] The contour points of each scanning column are determined, that is, only one contour point is retained on each scanning column. If there is a scanning column without a contour point, the cubic spline interpolation method is used to repair the contour, and finally a complete line structured light contour is obtained.

[0064] The line structured light contour extraction method of this embodiment has been successfully applied in the field of remote laser welding of metal parts to obtain the surface morphology of metal parts such as copper and aluminum, and provide height information for subsequent laser welding. Figure 2Taking the example line structured light original image shown as an example, the workpiece being measured is made of heat-resistant steel and pure copper, the surface morphology of the workpiece is complex, and there are multiple support rods around the workpiece, which can easily cause multiple reflections of the line structured light and form a pseudo contour on the workpiece surface. The line structured light contour extraction method of this embodiment can effectively improve the workpiece morphology measurement accuracy and anti-interference ability, and the workpiece detection misjudgment rate is reduced from 2.1% to 0.2%. The specific operation steps are as follows:

[0065] Step 1: The line structured light is incident at an angle, and the camera is located directly above the object to be measured to obtain the original line structured light image of the complex surface. Figure 2 As shown, due to the difference in the material of the workpiece surface, there is a dark area with low reflectivity (corresponding to label A); part of the structured light is reflected and irradiated onto the surface of the workpiece again, forming a pseudo contour (corresponding to label B).

[0066] Step 2: After preprocessing the image, scan it column by column and extract the line contour according to the set threshold. In this example, the initial threshold is 80, and no contour points are found in the dark area with low reflection. The false contour causes multiple contour points to appear in the corresponding scan column, such as Figure 3 As shown in the figure, C is a point without contour and D is a pseudo contour point. If there is only one contour point on the scan line, the contour point is pushed into the "reference pool". If there is no contour point on the scan line, it indicates that the contour of the area is missing. If there are multiple contour points on the scan line, it indicates that there are pseudo contour points.

[0067] Step 3: Reduce the threshold in steps of 10 and search again on the scan lines in the area where the contour is missing. The scans that have already found the contour points do not participate in this step. When all scan lines have found the contour points, or the threshold is reduced to the specified value, stop this step. In this example, after gradually reducing the threshold to 40, we get Figure 4 The contour shown in the figure, where E is a candidate contour point and F is a competing contour point. Point E in the figure is a contour point found after lowering the threshold and is pushed into the "candidate pool"; point F is multiple contour points that appear on a single scan line and are pushed into the "competing group". Although there are still a few scan lines where contour points have not been found, the set minimum threshold has been reached, so this step ends.

[0068] Step 4: Calculate the 8-neighborhood grayscale gradient of the points in the "candidate pool". If the grayscale gradient difference in each direction is small, the contour point is considered to be noise; if the grayscale gradient in one direction is significantly greater than that in other directions, the contour point is considered to belong to line structured light. If there are multiple points on a scanning column that meet the gradient directionality requirements at the same time, they are included in the "competition group".

[0069] Step 5: Calculate the point distance of all adjacent contour points in the reference pool as the basic entropy value of the line structured light contour; add all the "competition group" contour points on a certain scan column to the basic contour one by one, calculate the distance between adjacent points, and subtract them from the basic entropy value to select the contour point with the smallest entropy increase; then select other scan columns with multiple contour points and perform the same process; in this process, the selected contour points are not added to the "reference pool"; if a scan column has multiple "candidate" contour points, the contour points are selected according to this rule. Figure 5 , where G is the retained contour point and F is the deleted contour point.

[0070] Step 6: Determine the contour points of each scanning column according to the above rules, that is, only one contour point is retained on each scanning column. If there are still scanning columns without contour points, the cubic spline interpolation method is used to repair the contour, and finally a complete line structured light contour is obtained and converted into the height value of the workpiece surface, such as Figure 6 This method has good adaptive processing capabilities for local contour loss and interference, especially dark area contour loss caused by material reflectivity differences, and pseudo contours caused by structured light self-reflection.

[0071] In summary, this method does not need to increase the hardware of the line structured light measurement system. Through the design of structured light contour extraction strategy, it takes into account the integrity and accuracy of the contour. First, select high-quality contour points to build a reference pool; then, gradually lower the threshold of the weak reflective area to build a contour point candidate pool; for areas with multiple contour points, build a contour point competition group; determine whether to retain the contour points in the candidate pool based on the gradient direction difference; use the contour points in the reference pool to calculate the contour morphological entropy, and select the best contour points in each column of the competition group based on the principle of minimum entropy increase. According to the characteristics of line structured light images, this method designs a grouping processing mode by analyzing the idea of ​​manually identifying line structured light contours. It has good adaptability and can meet the height measurement of material surfaces with different degrees of non-uniformity. It has good anti-interference ability for structured light reflection interference caused by complex structures, fast calculation speed and high algorithm stability.

[0072] Example 2

[0073] This embodiment provides an electronic device, including: one or more processors and a memory, wherein the memory stores one or more programs, and the one or more programs include instructions for executing the complex surface structured light contour extraction method as described in Example 1.

[0074] In a typical configuration, an electronic device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0075] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0076] Example 3

[0077] This embodiment provides a computer-readable storage medium, including one or more programs for execution by one or more processors of an electronic device, wherein the one or more programs include instructions for executing the complex surface structured light contour extraction method as described in Example 1.

[0078] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0079] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A complex surface structured light contour extraction method, characterized in that: The steps include: Obtain the original line structured light image of the complex surface, and extract the pixel points in each column or row that are higher than a preset threshold as contour points in a column or row scanning manner; For a column or row with only one contour point, the contour point of the column or row is added to the reference pool; For columns or rows without contour points, the threshold is gradually lowered to scan until at least one contour point exists in the current column or row, or the threshold is lower than the preset minimum threshold; Calculating the distances between adjacent contour points in the reference pool to obtain a basic morphological entropy value of the structured light contour; For a column or row with multiple contour points, retain the contour point with the smallest entropy increment of the morphological entropy in the column or row; For columns or rows without contour points, interpolation is performed to ensure that there is at least one contour point on each column or each row, thereby obtaining the complex surface structured light contour.

2. The complex surface structure light contour extraction method according to claim 1, characterized in that: When the threshold is gradually lowered for scanning, the following steps are also included: If there is only one contour point in a column or row after lowering the threshold scan, the contour point is added to the candidate pool; The multi-neighborhood directional gradients of the contour points in the candidate pool are calculated, and contour points with similar directional gradients are screened out.

3. The complex surface structure light contour extraction method according to claim 2, characterized in that: Calculating the multi-neighborhood directional gradients of the contour points in the candidate pool and filtering out contour points with similar directional gradients comprises the following steps: Based on the color of the line structured light, a color difference map corresponding to the original image of the line structured light is obtained; For each contour point in the candidate pool, based on the color difference map, the gradient within the N neighborhood range is calculated, and the gradient value is searched according to the interval interval. When there is only one gradient value that is not in the reference interval and the other gradient values ​​are all in the reference interval, the distances between the gradient value point outside the reference interval and the maximum gradient value point and the minimum gradient value point in the reference interval are calculated respectively, and it is determined whether the distances are greater than the average spacing of the preset multiples in the reference interval. If so, the current contour point is retained. If not, the neighborhood range is expanded and the search is performed again. When the neighborhood range exceeds the preset value, the contour point is deleted.

4. The complex surface structure light contour extraction method according to claim 3, characterized in that: The interval is a preset value, or is updated according to a preset interval.

5. The complex surface structure light contour extraction method according to claim 1, characterized in that: The basic morphological entropy value is the cumulative sum of the distances between two adjacent contour points in the reference pool.

6. The complex surface structure light contour extraction method according to claim 1, characterized in that: The process of retaining the contour points with the smallest entropy increment in the morphological entropy within a column or row includes: The sum of the distances between each contour point in a column or row and the adjacent contour points on both sides in the reference pool when each contour point is added to the reference pool is calculated as the morphological entropy value, and the difference is made from the basic morphological entropy value to obtain the entropy increment of the morphological entropy.

7. The complex surface structure light contour extraction method according to claim 1, characterized in that: The interpolation is cubic spline interpolation.

8. The complex surface structure light contour extraction method according to claim 1, characterized in that: The scanning is line-by-line scanning or scanning at preset intervals.

9. An electronic device, characterized in that: include: One or more processors and a memory, wherein the memory stores one or more programs, and the one or more programs include instructions for executing the complex surface structured light contour extraction method as claimed in any one of claims 1-8.

10. A computer-readable storage medium, characterized in that: The method comprises one or more programs for execution by one or more processors of an electronic device, wherein the one or more programs comprise instructions for executing the complex surface structured light contour extraction method as claimed in any one of claims 1 to 8.

Images