Straight line inspection method and system for visual positioning

Abstract

This invention discloses a method and system for line inspection in visual positioning, relating to the field of image processing technology. The method includes: setting a search area based on the expected direction of the line; performing parallel binarization and line segmentation to obtain a binary image and a line segmentation image; performing contour filtering, candidate line segment fitting, and collinear connection on the line segmentation image to obtain a line connection image; extracting and fitting the longest contour to obtain the line endpoints; combining the binary image with system sampling along the line normal; calculating the line width through profile analysis and identifying defects such as broken lines, lines that are too thin, and lines that are too thick; finally, based on preset quality standards, quantitatively evaluating and automatically judging the effective length, average line width, and defect ratio. This method can filter reliable and complete line features for high-precision visual positioning, significantly improving the robustness and practicality of industrial vision systems in complex scenarios.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and more specifically to a method and system for line checking for visual positioning. Background Technology

[0002] In existing visual positioning technologies, the straight line, as an important geometric primitive, is widely used in workpiece pose estimation, guided alignment, and dimensional measurement due to its explicit mathematical expression and rich geometric constraints. To achieve efficient and accurate straight line extraction, researchers have proposed many classic algorithms, each with its own characteristics and limitations.

[0003] The Hough transform maps points in image space to parameter space (polar coordinate system). It detects straight lines by cumulative voting and has a certain robustness to discontinuities and noise. However, its accuracy is affected by the quantization of the parameter space and it can only detect straight lines of a fixed form.

[0004] Edge detection-based methods utilize edge extraction results, connect the edges, and fit them to obtain a straight line. This approach is suitable for scenarios where edge extraction is required before analyzing the features of the straight line. Although it can achieve sub-pixel accuracy, it depends on edge quality and is quite sensitive to noise.

[0005] LSD (Line Segment Detector) is based on image gradients and uses region growing and rectangular approximation. It is often used for natural scenes and document analysis. It does not require parameter tuning and has a fast detection speed, but it is sensitive to blurred edges and may produce broken line segments.

[0006] These traditional methods are suitable for scenarios with prominent line features, clean backgrounds, and low requirements for absolute accuracy. However, for visual positioning tasks in industrial inspection that require precise quantification of line width, judgment of microscopic continuity, and adaptation to complex imaging conditions, they are often insufficient as a complete solution. Therefore, there is an urgent need for a detection method that can reliably extract high-quality line segments from complex industrial scenes and assess their reliability, providing more reliable feature inputs for high-precision visual positioning. Summary of the Invention

[0007] This invention provides a method for checking straight lines for visual positioning, which solves the technical problem that traditional straight line detection algorithms cannot perform quantitative assessment of the physical quality of straight lines in complex industrial scenarios.

[0008] This invention is achieved through the following technical solution:

[0009] In a first aspect, a method for checking straight lines for visual positioning includes the following steps:

[0010] Based on the expected direction of the line segment to be detected, a rectangular search region is set in the image to be detected, and the image within the rectangular search region is binarized and segmented into linear objects to obtain a binary image and a line segmentation image.

[0011] The line segmentation image is subjected to contour filtering, and each of the filtered contours is fitted to generate a candidate line segment set.

[0012] Based on the included angle and endpoint distance between each line segment in the candidate line segment set, line segments belonging to the same straight line are determined and connected to generate a line connection image;

[0013] Find the longest contour in the line connection image and perform straight line fitting to obtain the coordinates of the endpoints of the straight line segment;

[0014] Line width analysis is performed on the binary image and the endpoint coordinates of the acquired straight line segments to obtain line width data and defect information;

[0015] Based on the obtained line width data and defect information, determine whether the straight line segment to be inspected is qualified.

[0016] Further, the binarization processing of the image within the rectangular search area includes:

[0017] The binary image is generated by processing the image within the rectangular search area using the Sauvola local adaptive binarization method based on integral image acceleration.

[0018] Further, the line segmentation image is subjected to contour filtering, and each filtered contour is fitted as a candidate line segment, including:

[0019] Obtain all contours in the line segmentation image and calculate the bounding rectangle of each contour;

[0020] Filter out contours whose long side length is greater than the first threshold and whose short side length is less than the second threshold;

[0021] The least squares method is used to fit each selected contour to obtain the corresponding candidate line segment.

[0022] Furthermore, the step of determining and connecting line segments belonging to the same straight line based on the included angle and endpoint distance between each line segment in the candidate line segment set specifically involves:

[0023] When the angle between the first line segment and the second line segment in the candidate line segment set is less than the angle threshold, and the minimum distance between the endpoints of the two line segments is less than the connection distance threshold, it is determined that the first line segment and the second line segment belong to the same straight line, and their corresponding contours are connected in the line segmentation image.

[0024] Further, the step of finding the longest contour in the line connection image and performing line fitting to obtain the endpoint coordinates of the line segment includes:

[0025] Obtain the gradient direction and gradient magnitude of each contour point on the longest contour;

[0026] Contour points are selected where the difference between the gradient direction and the expected direction of the line segment is less than the angle tolerance, and the gradient magnitude is greater than the magnitude threshold.

[0027] The least squares method is used to fit a straight line to the selected set of contour points to obtain the coordinates of the endpoints of the straight line segment.

[0028] Furthermore, the linewidth analysis includes:

[0029] Based on the endpoint coordinates of the line segment, calculate the unit direction vector and normal vector of the line;

[0030] Multiple sampling points are evenly distributed along a straight line;

[0031] For each sampling point, scan along its positive and negative normal directions in the binary image to obtain the coordinates of the left and right boundary points;

[0032] Based on the completeness of the acquired boundary points and the distance between the boundary points, determine whether the current sampling point is a normal line segment, an excessively thin defect, an excessively thick defect, or a broken line defect.

[0033] Collect data from each sampling point and calculate the average line width, effective length, and length of various defects of the straight line segment.

[0034] Furthermore, the step of determining whether the straight line segment to be inspected is qualified based on the obtained line width data and defect information specifically involves:

[0035] A comprehensive judgment is made based on whether the effective length of the straight line segment is within a preset length range, whether the average line width is within a preset line width range, and whether the length ratio of various defects exceeds a preset defect tolerance.

[0036] Furthermore, the linear object segmentation employs the Steger algorithm based on the second-order directional derivative of the image and Hessian matrix analysis to locate the center of the linear object with sub-pixel precision and obtain the line segmentation image.

[0037] Secondly, this application provides a line checking system for visual positioning, used to implement the line checking method for visual positioning as described above; the system includes:

[0038] The image acquisition and preprocessing module is used to set a rectangular search region in the image to be detected according to the expected direction of the line segment to be detected, and to perform binarization processing and line object segmentation on the image within the rectangular search region to obtain a binary image and a line segmentation image.

[0039] The candidate line segment generation module is communicatively connected to the image acquisition and preprocessing module. It is used to perform contour filtering on the line segmentation image and fit each of the filtered contours to generate a candidate line segment set.

[0040] The line segment connection module is communicatively connected to the candidate line segment generation module. It is used to determine and connect line segments belonging to the same straight line to generate a line connection image based on the included angle and endpoint distance between each line segment in the candidate line segment set.

[0041] The precise positioning module is communicatively connected to the line segment connection module and is used to find the longest contour in the line connection image and perform straight line fitting to obtain the endpoint coordinates of the straight line segment.

[0042] The line width and defect analysis module is communicatively connected to the image acquisition and preprocessing module and the precise positioning module, respectively, and is used to perform line width analysis on the binary image and the endpoint coordinates of the acquired straight line segment to obtain line width data and defect information.

[0043] The pass / fail determination module is communicatively connected to the line width and defect analysis module, and is used to determine whether the straight line segment to be inspected is qualified based on the obtained line width data and defect information.

[0044] Thirdly, this application provides a computer-readable storage medium storing a line checking program for visual positioning, wherein when the line checking program for visual positioning is executed by a processor, it implements the steps of the line checking method for visual positioning as described above.

[0045] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0046] By innovatively introducing a line width and defect analysis module based on binary images at the end of the traditional straight line inspection process, this method can accurately quantify the physical properties of straight lines and systematically identify and locate various micro-defects such as broken lines, lines that are too thin, or lines that are too thick. It can not only output accurate coordinates of the endpoints of straight lines, but also automatically judge the integrity and uniformity of straight lines according to preset quality standards. Ultimately, it can select truly reliable and usable straight line features for high-precision visual positioning tasks, thereby significantly improving the robustness and reliability of industrial inspection and positioning systems when facing complex imaging conditions and defective workpieces. Attached Figure Description

[0047] To more clearly illustrate the technical solutions of the exemplary embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly described below. It should be understood that the following drawings only show some embodiments of the present invention and should not be considered as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort. In the drawings:

[0048] Figure 1 A flowchart of a line checking method for visual positioning provided in an embodiment of this application;

[0049] Figure 2 This is a schematic diagram illustrating the positional relationship between the rectangular search area and the line to be inspected, provided in an embodiment of this application.

[0050] Figure 3 A schematic diagram of a binary image obtained after Sauvola binarization processing, as provided in an embodiment of this application.

[0051] Figure 4 This is a schematic diagram of a line segmentation image obtained by processing the Steger algorithm, provided in an embodiment of this application.

[0052] Figure 5 A schematic diagram of a line connection image generated after connecting candidate line segments, provided in an embodiment of this application;

[0053] Figure 6 A schematic diagram illustrating the principle of linewidth analysis along the normal direction of a straight line, provided for an embodiment of this application;

[0054] Figure 7 A schematic diagram illustrating the linewidth and defect analysis results provided in an embodiment of this application;

[0055] Figure 8 A schematic diagram showing the annotations of straight line endpoints and various defect locations in an image, provided for embodiments of this application;

[0056] Figure 9 A complete and defect-free schematic diagram of the straight line to be inspected provided in Embodiment 1 of this application;

[0057] Figure 10 This is a schematic diagram of a straight line under inspection with fracture defects, provided in Embodiment 2 of this application.

[0058] Figure 11 This refers to the rectangular search area image defined for the broken straight line in Embodiment 2 of this application;

[0059] Figure 12 This is the result image obtained after binarizing the rectangular search area image in Embodiment 2 of this application;

[0060] Figure 13This is the result image obtained by performing Steger line segmentation on the rectangular search region image in Embodiment 2 of this application;

[0061] Figure 14 This is a diagram showing the result obtained after filtering and connecting the segmentation results in Embodiment 2 of this application;

[0062] Figure 15 This is a schematic diagram of the detection results of a straight line with the location of the broken line defect marked in Embodiment 2 of this application. Detailed Implementation

[0063] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the embodiments and accompanying drawings. The illustrative embodiments and descriptions of the present invention are only used to explain the present invention and are not intended to limit the present invention.

[0064] First, some of the technical terms used in this application will be explained to help those skilled in the art understand this application.

[0065] Roi: Region of Interest;

[0066] LSD: Line Segment Detector;

[0067] API: Application Programming Interface;

[0068] OTSU: Otsu's method, the method of maximum inter-class variance.

[0069] Firstly, such as Figure 1 As shown, this application provides a method for line checking for visual positioning, including the following steps:

[0070] Step S1: Based on the expected direction of the line segment to be detected, set a rectangular search region in the image to be detected, and perform binarization and line object segmentation on the image within the rectangular search region to obtain a binary image and a line segmentation image.

[0071] Step S2: Perform contour filtering on the line segmentation image, and fit each filtered contour as a candidate line segment to generate a candidate line segment set;

[0072] Step S3: Based on the included angle and endpoint distance between each line segment in the candidate line segment set, determine and connect line segments belonging to the same straight line to generate a line connection image;

[0073] Step S4: Find the longest contour in the line connection image and perform straight line fitting to obtain the coordinates of the endpoints of the straight line segment;

[0074] Step S5: Perform line width analysis on the binary image and the obtained endpoint coordinates of the straight line segment to obtain line width data and defect information;

[0075] Step S6: Based on the obtained line width data and defect information, determine whether the straight line segment to be inspected is qualified.

[0076] This embodiment obtains a basic image by setting a rectangular search area and combining binarization and linear object segmentation. Then, it optimizes broken or noisy straight lines through contour filtering, line segment fitting and connection. Finally, it accurately locates the endpoints by fitting the longest contour and extracts defect data based on line width analysis. Ultimately, it realizes the automated judgment of the integrity and quality of the straight line segment to be inspected, thereby significantly improving the robustness, accuracy and efficiency of straight line detection in visual positioning.

[0077] In one embodiment, step S1: Based on the expected direction of the line segment to be detected, a rectangular search region is set in the image to be detected, and the image within the rectangular search region is subjected to binarization processing and line object segmentation to obtain a binary image and a line segmentation image, specifically including the following steps:

[0078] Step S11: Based on the known expected direction of the line segment to be detected, set a rectangular search region Roi in the image to be detected, whose long side direction is parallel to the expected direction, such as... Figure 2 As shown; where the known expected direction refers to the approximate direction of the straight line to be inspected, which is known through prior information before the inspection is carried out; the prior information is such as CAD drawings, standard workpiece pose, or the results of the previous inspection;

[0079] Step S12: For the image within the rectangular search region Roi, the Sauvola local adaptive binarization method is used to process the image within the rectangular search region to generate the binary image binary;

[0080] Specifically, integral image technology is used to accelerate the local mean. With local standard deviation The calculation involves pre-calculating the integral image of the original image. sum of square integrals graph :

[0081] Equation (1)

[0082] Equation (2)

[0083] in, These are the original image pixel values;

[0084] The integral image of the original image is obtained based on the above calculations. sum of square integrals graph It can quickly calculate the sum of pixels and the sum of squares within any rectangular window, thus providing an efficient solution. and ;in, Therefore Center, window Local mean within; Therefore Center, window Local standard deviation within;

[0085] Based on calculations and For each pixel of the image within the rectangular search area. Calculate the dynamic threshold based on its local neighborhood. As shown in the following formula:

[0086] Equation (3)

[0087] in, For pixels The local threshold at that point serves as the boundary value for binarization decision-making; The parameters are adjusted to control the local dynamic threshold in the Sauvola algorithm. Local grayscale standard deviation Sensitivity; Here, is the normalization parameter, and is the expected maximum value of the local grayscale standard deviation, used to calculate the actual local standard deviation. Standardize;

[0088] Based on local threshold Binarize each pixel to obtain a binary image. , where any pixel coordinate The pixel value is defined as:

[0089] Equation (4)

[0090] in, For binarized images in grayscale value at that location For the original image in The grayscale value at the specified location; the binary image obtained after this step is as follows: Figure 3 As shown;

[0091] Step S13: For the image within the rectangular search area, the Steger algorithm based on Hessian matrix analysis is used to segment linear objects to generate the line segmentation image;

[0092] Specifically, the Steger algorithm determines the normal direction of the lines by analyzing the second directional derivative of the lines in the image, and performs Taylor expansion in the normal direction to locate the center point of the lines with sub-pixel precision, thereby obtaining the line segmentation image.

[0093] More specifically, the local gray-level curvature of each pixel in the image is characterized by calculating the Hessian matrix of each pixel, which is composed of all the second-order directional derivatives of the image;

[0094] Perform eigenvalue analysis on the matrix and determine the direction of the eigenvector corresponding to the largest eigenvalue (absolute value) as the local normal direction of the line at that point;

[0095] By performing a Taylor expansion of the grayscale distribution along this one-dimensional normal direction, and solving the equation where the first derivative of the grayscale is zero, the center position of the line with sub-pixel precision can be calculated.

[0096] Aggregate all located sub-pixel center points to generate a binary image representing the line skeleton structure, i.e., a line segmentation image, such as... Figure 4 As shown.

[0097] Based on step S13, high-precision, sub-pixel-level extraction of line centers is achieved through second-derivative analysis and sub-pixel localization, effectively overcoming the problems of traditional edge detection being sensitive to noise and having localization accuracy limited to the pixel level. The generated line segmentation image can clearly and continuously express the skeletal structure of the lines, providing a high-quality and highly consistent input foundation for subsequent line segment fitting, connection, and accurate analysis.

[0098] In one embodiment, step S2: performing contour filtering on the line segmentation image, and fitting each filtered contour to generate a candidate line segment set, specifically includes the following steps:

[0099] Step S21: Extract all contours in the line segmentation image, and calculate the length of the long side and the length of the short side for each contour by its circumscribed rotation rectangle;

[0100] Step S22: Based on the preset size threshold, filter out contours whose long side length is greater than the long side threshold long_thre and whose short side length is less than the short side threshold short_thre;

[0101] Step S23: For each contour retained after filtering, use the least squares method to perform line fitting to generate a set of candidate line segments.

[0102] In one embodiment, step S3: Based on the included angle and endpoint distance between each line segment in the candidate line segment set, determine and connect line segments belonging to the same straight line to generate a line connection image, specifically including the following steps:

[0103] Step S31: From the candidate line segment set, select any two different line segments as the first and second line segments to be judged;

[0104] Step S32: Calculate the included angle between the first line segment and the second line segment, and the minimum distance between the endpoints of the first line segment and the second line segment;

[0105] Step S33: Determine whether the included angle is less than a preset angle threshold angle_thre, and whether the minimum distance between the endpoints is less than a preset connection distance threshold connect_dis;

[0106] Step S34: If the included angle is less than a preset angle threshold angle_thre, and the minimum distance between the endpoints is less than a preset connection distance threshold, then it is determined that the first line segment and the second line segment belong to the same straight line;

[0107] Step S35: In the line segmentation image, connect the contours corresponding to line segments that are determined to belong to the same straight line;

[0108] Step S36: Repeat steps S31 to S35 until all the line segments that need to be connected in the candidate line segment set have been processed, generating the final line connection image, such as... Figure 5 As shown.

[0109] This step, through geometric relationships (angle and distance), intelligently connects short line segments separated by local breaks, noise, or occlusion, thereby restoring the integrity of the straight line. This significantly enhances the algorithm's robustness to discontinuous and fragmented straight line features.

[0110] In one embodiment, step S4: finding the longest contour in the line connection image and performing straight line fitting to obtain the endpoint coordinates of the straight line segment, specifically includes the following steps:

[0111] Step S41: Find and extract the longest contour in the line connection image, and calculate the gradient direction and gradient magnitude of each contour point on the longest contour; wherein, the gradient direction is the edge normal direction;

[0112] Specifically, for contour points Calculate its value in the original image middle and gradient of direction , Then the gradient magnitude and gradient direction for:

[0113] Equation (5)

[0114] Equation (6)

[0115] Specifically, the gradient direction Rotate 90 degrees to obtain the tangent direction of the edge. That is, the outline direction, and normalized to Interval:

[0116] Equation (7)

[0117] Step S42: According to the preset expected edge normal direction With gradient magnitude threshold Filter the contour points to obtain the filtered point set. ;

[0118] Specifically, calculate the actual direction. in line with expected direction minimum difference :

[0119] Equation (8)

[0120] Only retain those that simultaneously satisfy and The contour points form a point set for fitting. ,in Preset angle tolerance;

[0121] Step S43: Apply the least squares method to the filtered point set. By performing line fitting, the starting and ending points of the line segment can be obtained. and termination endpoint .

[0122] This step uses gradient direction filtering to eliminate a large number of noise points or outliers that are inconsistent with the direction of the target line, providing a high-quality and highly consistent set of input points for least squares fitting, which significantly improves the accuracy and anti-interference ability of fitting the endpoints of the line.

[0123] In one embodiment, such as Figure 6 As shown, step S5: Perform line width analysis on the binary image and the endpoint coordinates of the acquired straight line segment to obtain line width data and defect information, specifically including the following steps:

[0124] Step S51: Based on the starting endpoint of the line segment and termination endpoint Calculate the unit direction vector of the line. and unit normal vector ;

[0125] Step S52: Along the unit direction vector N sampling points are uniformly set on the straight line segment, and the coordinates of the i-th sampling point are... The following was obtained through linear interpolation:

[0126] Equation (9)

[0127] Equation (10)

[0128] In the formula, These are linear interpolation parameters; For sampling point index; The total number of sampling points is specified as the length of the line segment; For the first The coordinates of each sampling point;

[0129] Step S53: For each sampling point Along its normal vector The right boundary point is located and recorded by scanning the binary image in both positive and negative directions. and left boundary point The coordinates.

[0130] Step S54: For each sampling point, based on the existence of the recorded left and right boundary points and the distance between them, determine and record the line segment status and position information at that point.

[0131] Step S54A: If the left boundary point and right boundary point If both exist, then the Euclidean distance between the two points is calculated according to the following formula and used as the line width at the current point. and the preset line width threshold range Compare them.

[0132] Equation (11)

[0133] like The condition is judged to be normal, and the effective length of the straight line is [value missing]. Add 1 cumulatively;

[0134] like The defect is judged to be excessively thin; the effective length of the straight line is... Cumulative increment of 1, excessively fine defect length Increment by 1 and record the location of the defect center. ;

[0135] like The defect is judged to be excessively thick; the effective length of the straight line is... Add 1 cumulatively to the length of the excessively coarse defect. Increment by 1 and record the location of the defect center. ;

[0136] Step S54B: If at least one of the left and right boundary points is missing, the condition is determined to be a broken wire defect. The length of the broken wire defect is... Increment by 1 and record the coordinates of the sampling point. As the location of the defect.

[0137] Step S55: After traversing all sampling points, based on the judgment results and recording results, calculate the effective length of the line segment. Average line width And the cumulative length of various defects, such as the length of a broken wire defect. Excessive length of defect excessively fine defect length .

[0138] Among them, the average line width The calculation formula is:

[0139] Equation (12)

[0140] in, To determine the effective number of measurement points, For the first Line width of each valid measurement point; a valid measurement point is a sampling point that exists at both the left and right boundary points;

[0141] Example of linewidth and defect analysis results: Figure 7 As shown;

[0142] Step S56: Integrate the various data obtained from statistical calculations with the recorded defect location information to obtain line width data and defect information.

[0143] Specifically, this step outputs a consolidated result containing the following key information:

[0144] Fitted start endpoints With the termination endpoint Coordinates, total length of the line This yields the geometric properties of the line;

[0145] Effective length of straight line Average line width of the entire segment Obtain line width statistics.

[0146] Total length of wire breakage defects Total length of excessively coarse defects Total length of excessively fine defects To obtain quantitative statistical information on defects;

[0147] Record the specific location coordinates of each type of defect in step S54. For example, the coordinates of the break point are the sampling point coordinates. The location of the defect center is either too coarse or too fine. This process involves obtaining defect location information. Finally, the endpoints of the lines and the identified defects can be labeled on the original image, such as... Figure 8 As shown.

[0148] This embodiment achieves precise quantification of the physical properties of a straight line by combining geometric endpoints with binary images for normal profile analysis. It can also systematically identify, locate, and statistically analyze various microscopic defects such as broken lines, lines that are too thin, or lines that are too thick. This provides objective and comprehensive evaluation data for the physical quality of the straight line, surpassing the limitations of traditional methods that only provide geometric information.

[0149] In one embodiment, step S6: Based on the obtained line width data and defect information, determine whether the straight line segment to be inspected is qualified, specifically including the following steps:

[0150] Step S61: Based on the line width data and defect information, perform the following three parallel acceptance / disapproval checks:

[0151] Determine the effective length of the straight line segment Does it meet the following requirements: ;

[0152] Determine the average line width of the line segment Does it meet the following requirements: ;

[0153] Calculate the total length of the straight line segment And determine the proportion of broken wire defects respectively. The proportion of excessively coarse defects and the proportion of excessively fine defects Whether it exceeds the corresponding preset defect tolerance.

[0154] Step S62: The straight line segment is deemed qualified if and only if the effective length, average line width, and defect ratio of all categories all meet their respective preset conditions;

[0155] If the effective length, average line width, and defect ratio of all categories all meet their respective preset conditions, but any one of these conditions is not met, then the straight line segment is deemed unqualified.

[0156] This step enables automated quality assessment based on multi-dimensional quantitative indicators such as length, line width, and various defects. According to preset industrial standards, it can screen out truly reliable and usable straight line features for subsequent high-precision visual positioning tasks, thereby improving the decision-making intelligence and reliability of the entire system.

[0157] The present invention will now be described with reference to specific embodiments. In these embodiments, complete, defect-free straight lines and straight lines with local breaks were selected as test objects, such as... Figure 9 and Figure 10 As shown, this is to verify the detection and evaluation capabilities of the method of the present invention under different conditions.

[0158] Example 1: Targeting Figure 9 The inspection is performed on a straight line that is complete and without defects.

[0159] Specifically, a rectangular search region is defined based on the direction of the straight line. Sauvola binarization and Steger linear object segmentation, accelerated using integral images, are sequentially performed on the image of this region to obtain a binary image for linewidth analysis and a line segmentation image representing the line skeleton, respectively. After contour filtering, line segment fitting, and connection, gradient direction filtering is applied to the final longest contour point set, and the endpoint coordinates of the straight line are accurately fitted using the least squares method. Subsequently, normal scanning and linewidth analysis are performed based on the binary image and endpoint coordinates.

[0160] The results showed that the effective length of the straight line was 1675, the average line width was 10.33, and no defects such as broken lines, excessive thickness or thinness were detected. All indicators were within the preset tolerance range.

[0161] Therefore, the line is deemed qualified, and its endpoint coordinate information can be directly used for subsequent high-precision visual positioning operations.

[0162] Example 2: This example demonstrates the invention's ability to handle straight lines with broken lines, as shown in the example. Figure 10 As shown.

[0163] The testing process is as follows:

[0164] Performed within the designated Roi area, such as Figure 11 As shown, after the same preprocessing, binary images and line-segmented images are obtained, as shown below. Figure 12 and Figure 13 As shown in the figure. In the online segmentation results, due to breaks, the straight line is initially identified as multiple short line segments. This invention successfully connects the broken parts belonging to the same straight line by calculating the included angle and the distance between the endpoints of the line segments, as shown in the figure. Figure 14 As shown.

[0165] After performing the same fine-fitting and linewidth analysis on the connected contour, the results showed that the effective length of the line segment was 1650, the average linewidth was 10.06, but a broken line defect with a length of 25 was detected. Calculations showed that the broken line defect accounted for approximately 1.5% of the effective length, which did not exceed the set tolerance threshold of 5%.

[0166] Therefore, although the straight line has defects, it can still be determined to meet the usage requirements and can be used for subsequent positioning, but its quality assessment result is slightly lower than that of the intact straight line in Example 1. Finally, the detected breakage location is... Figure 15 Label it visually.

[0167] In summary, through a comparison of the two embodiments, it is verified that the method of the present invention can not only extract the endpoints of a straight line with high precision, but also quantitatively evaluate its physical integrity and dimensional consistency, thereby providing a reliable feature input for the visual positioning system to intelligently select it. The technical effects of this application are as follows:

[0168] In the image preprocessing stage, the computational efficiency is significantly optimized by introducing integral image technology into the Sauvola local adaptive binarization algorithm. Traditional Sauvola algorithms require repeatedly traversing the neighborhood window to calculate the mean and standard deviation when calculating the local threshold for each pixel, resulting in high time complexity. This invention, by pre-calculating the integral image, can quickly obtain the pixel sum and sum of squares of any rectangular region with constant time complexity, thereby reducing the total time complexity of local threshold calculation from... Down to This greatly accelerates the processing speed and provides the necessary efficiency guarantee for real-time detection.

[0169] In the line segment generation and optimization stage, this invention enhances the algorithm's robustness to incomplete or noisy features through two key operations. First, after obtaining the candidate line segment set, intelligent connection is performed based on the included angle and endpoint distance between the line segments. This step, by setting included angle thresholds and connection distance thresholds, can reconnect short line segments that are separated due to local breaks, imaging noise, or occlusion but actually belong to the same straight line, thereby restoring the structural integrity of the straight line and enabling the algorithm to handle straight line features with discontinuities.

[0170] In the fine fitting stage of the line endpoints, an innovative point set optimization method based on gradient direction filtering was introduced. Before fitting the line using the least squares method, the gradient direction of each point on the contour was calculated, and points whose gradient direction is consistent with the expected line normal direction and whose gradient magnitude is large enough were selected. This filtering operation effectively eliminated a large number of noise points, outliers, or edge points belonging to other structures with inconsistent directions, providing a high-quality and highly consistent input point set for least squares fitting, thereby significantly improving the fitting accuracy and anti-interference ability of the final line endpoint coordinates.

[0171] This method introduces a comprehensive linewidth and defect analysis that combines geometric and grayscale information. It not only outputs the endpoint coordinates of the line but also performs dense sampling and profile analysis along the normal direction on the fitted line segment based on a Sauvola binarized image. By accurately calculating the physical linewidth at each sampling point, it can objectively identify and quantify various microscopic defects such as "broken lines," "too thin," and "too thick," while recording the specific location of the defects. By statistically analyzing indicators such as effective length, average linewidth, and cumulative length of various defects, it achieves a multi-dimensional quantitative evaluation of the physical quality of the line, providing a direct and reliable decision-making basis for determining whether the line's features can be used for high-precision visual positioning.

[0172] Secondly, this application provides a line checking system for visual positioning, used to implement the line checking method for visual positioning as described above; the system includes:

[0173] The image acquisition and preprocessing module is used to set a rectangular search region in the image to be detected according to the expected direction of the line segment to be detected, and to perform binarization processing and line object segmentation on the image within the rectangular search region to obtain a binary image and a line segmentation image.

[0174] The candidate line segment generation module is communicatively connected to the image acquisition and preprocessing module. It is used to perform contour filtering on the line segmentation image and fit each of the filtered contours to generate a candidate line segment set.

[0175] The line segment connection module is communicatively connected to the candidate line segment generation module. It is used to determine and connect line segments belonging to the same straight line to generate a line connection image based on the included angle and endpoint distance between each line segment in the candidate line segment set.

[0176] The precise positioning module is communicatively connected to the line segment connection module and is used to find the longest contour in the line connection image and perform straight line fitting to obtain the endpoint coordinates of the straight line segment.

[0177] The line width and defect analysis module is communicatively connected to the image acquisition and preprocessing module and the precise positioning module, respectively, and is used to perform line width analysis on the binary image and the endpoint coordinates of the acquired straight line segment to obtain line width data and defect information.

[0178] The pass / fail determination module is communicatively connected to the line width and defect analysis module, and is used to determine whether the straight line segment to be inspected is qualified based on the obtained line width data and defect information.

[0179] The functions of each module in the above-mentioned line inspection system for visual positioning correspond to the steps in the above-mentioned line inspection method embodiment for visual positioning, and their functions and implementation processes will not be described in detail here.

[0180] Thirdly, embodiments of this application provide a line inspection device for visual positioning. The line inspection device for visual positioning can be a personal computer (PC), a laptop computer, a server, or other device with data processing capabilities.

[0181] In this embodiment, the line inspection device for visual positioning may include a processor, a memory, a communication interface, and a communication bus.

[0182] The communication bus can be of any type and is used to interconnect the processor, memory, and communication interface.

[0183] The communication interface includes input / output (I / O) interfaces, physical interfaces, and logical interfaces used for interconnecting internal components of the line inspection equipment for visual positioning, as well as interfaces used for interconnecting the line inspection equipment for visual positioning with other devices (such as other computing devices or user equipment). Physical interfaces can be Ethernet interfaces, fiber optic interfaces, ATM interfaces, etc.; user equipment can be displays, keyboards, etc.

[0184] Memory can be various types of storage media, such as random access memory (RAM), read-only memory (ROM), non-volatile RAM (NVRAM), flash memory, optical storage, hard disk, programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), etc.

[0185] The processor can be a general-purpose processor, which can call a line checking program for visual positioning stored in memory and execute the line checking method for visual positioning provided in the embodiments of this application. For example, the general-purpose processor can be a central processing unit (CPU). The method executed when the line checking program for visual positioning is called can be referred to the various embodiments of the line checking method for visual positioning in this application, and will not be repeated here.

[0186] Fourthly, embodiments of this application also provide a readable storage medium.

[0187] The present application has a readable storage medium storing a line checking program for visual positioning, wherein when the line checking program for visual positioning is executed by a processor, it implements the steps of the line checking method for visual positioning as described above.

[0188] The method implemented when the line checking procedure for visual positioning is executed can be referred to in the various embodiments of the line checking method for visual positioning in this application, and will not be repeated here.

[0189] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for checking straight lines in visual positioning, characterized in that, Includes the following steps: Based on the expected direction of the line segment to be detected, a rectangular search region is set in the image to be detected, and the image within the rectangular search region is binarized and segmented into linear objects to obtain a binary image and a line segmentation image. The line segmentation image is subjected to contour filtering, and each of the filtered contours is fitted to generate a candidate line segment set. Based on the included angle and endpoint distance between each line segment in the candidate line segment set, line segments belonging to the same straight line are determined and connected to generate a line connection image; Find the longest contour in the line connection image and perform straight line fitting to obtain the coordinates of the endpoints of the straight line segment; Line width analysis is performed on the binary image and the endpoint coordinates of the acquired straight line segments to obtain line width data and defect information; Based on the obtained line width data and defect information, determine whether the straight line segment to be inspected is qualified.

2. The method for line inspection for visual positioning according to claim 1, characterized in that, The binarization process of the image within the rectangular search area includes: The binary image is generated by processing the image within the rectangular search area using the Sauvola local adaptive binarization method based on integral image acceleration.

3. The method for line inspection for visual positioning according to claim 1, characterized in that, The line segmentation image is subjected to contour filtering, and each filtered contour is fitted as a candidate line segment, including: Obtain all contours in the line segmentation image and calculate the bounding rectangle of each contour; Filter out contours whose long side length is greater than the first threshold and whose short side length is less than the second threshold; The least squares method is used to fit each selected contour to obtain the corresponding candidate line segment.

4. The method for line inspection for visual positioning according to claim 1, characterized in that, The step of determining and connecting line segments belonging to the same straight line based on the included angle and endpoint distance between each line segment in the candidate line segment set specifically involves: When the angle between the first line segment and the second line segment in the candidate line segment set is less than the angle threshold, and the minimum distance between the endpoints of the two line segments is less than the connection distance threshold, it is determined that the first line segment and the second line segment belong to the same straight line, and their corresponding contours are connected in the line segmentation image.

5. The method for line inspection for visual positioning according to claim 1, characterized in that, The step of finding the longest contour in the line connection image and performing line fitting to obtain the endpoint coordinates of the line segment includes: Obtain the gradient direction and gradient magnitude of each contour point on the longest contour; Contour points are selected where the difference between the gradient direction and the expected direction of the line segment is less than the angle tolerance, and the gradient magnitude is greater than the magnitude threshold. The least squares method is used to fit a straight line to the selected set of contour points to obtain the coordinates of the endpoints of the straight line segment.

6. The method for line checking for visual positioning according to claim 1, characterized in that, The linewidth analysis includes: Based on the endpoint coordinates of the line segment, calculate the unit direction vector and normal vector of the line; Multiple sampling points are evenly distributed along a straight line; For each sampling point, scan along its positive and negative normal directions in the binary image to obtain the coordinates of the left and right boundary points; Based on the completeness of the acquired boundary points and the distance between the boundary points, determine whether the current sampling point is a normal line segment, an excessively thin defect, an excessively thick defect, or a broken line defect. Collect data from each sampling point and calculate the average line width, effective length, and length of various defects of the straight line segment.

7. The method for line inspection for visual positioning according to claim 6, characterized in that, The step of determining whether the straight line segment to be inspected is qualified based on the obtained line width data and defect information is as follows: A comprehensive judgment is made based on whether the effective length of the straight line segment is within a preset length range, whether the average line width is within a preset line width range, and whether the length ratio of various defects exceeds a preset defect tolerance.

8. The method for line checking for visual positioning according to claim 1, characterized in that, The linear object segmentation employs the Steger algorithm, based on the second-order directional derivative of the image and Hessian matrix analysis, to locate the center of the linear object with sub-pixel precision and obtain the segmented image.

9. A straight line inspection system for visual positioning, characterized in that, A system for implementing the line checking method for visual positioning according to any one of claims 1-8; the system comprises: The image acquisition and preprocessing module is used to set a rectangular search region in the image to be detected according to the expected direction of the line segment to be detected, and to perform binarization processing and line object segmentation on the image within the rectangular search region to obtain a binary image and a line segmentation image. The candidate line segment generation module is communicatively connected to the image acquisition and preprocessing module. It is used to perform contour filtering on the line segmentation image and fit each of the filtered contours to generate a candidate line segment set. The line segment connection module is communicatively connected to the candidate line segment generation module. It is used to determine and connect line segments belonging to the same straight line to generate a line connection image based on the included angle and endpoint distance between each line segment in the candidate line segment set. The precise positioning module is communicatively connected to the line segment connection module and is used to find the longest contour in the line connection image and perform straight line fitting to obtain the endpoint coordinates of the straight line segment. The line width and defect analysis module is communicatively connected to the image acquisition and preprocessing module and the precise positioning module, respectively, and is used to perform line width analysis on the binary image and the endpoint coordinates of the acquired straight line segment to obtain line width data and defect information. The pass / fail determination module is communicatively connected to the line width and defect analysis module, and is used to determine whether the straight line segment to be inspected is qualified based on the obtained line width data and defect information.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a line checking program for visual positioning, wherein when the line checking program for visual positioning is executed by a processor, it implements the steps of the line checking method for visual positioning as described in any one of claims 1 to 8.

Images