Method for establishing linear edge space equation on finite plane and edge detection method

Abstract

The invention provides a method for establishing a linear edge space equation on a limited plane and an edge detection method.The method comprises the steps that firstly, a limited plane image in a specific scene is obtained through shooting, linear edge detection of depth information is carried out based on the depth information of the limited plane and the ground stored in a pixel matrix, and linear edge detection is carried out based on the edge detection result; establishing a finite plane depth equation, and calculating a depth equation coefficient; carrying out edge detection on the image based on a Canny algorithm; judging and selecting a corresponding detection result on the basis of an edge variance detected by image information and an edge variance detected by depth, and finally establishing a linear edge space equation on the basis of pixel coordinates of the obtained linear edge detection result and a finite plane depth equation. According to the method, the image information and the depth information are fused, the advantages of the image information and the depth information are fully utilized, the accuracy and robustness of linear edge detection are improved, and the method can stably operate in various complex scenes.

Description

Technical Field

[0001] This invention belongs to the field of image edge detection technology based on computer vision, and particularly relates to a method for establishing the spatial equation of straight line edges on a finite plane and an edge detection method. Background Technology

[0002] With the continuous development of computer vision technology, image edge detection has been widely applied in many fields, such as industrial inspection, autonomous driving, and medical image processing. Line edge detection, as an important branch of edge detection, is of great significance for tasks such as object contour recognition and shape analysis. Traditional line edge detection methods mainly include image gradient-based operator detection methods (such as Sobel and Canny), Hough transform methods, and deep learning-based detection methods. In recent years, multi-information fusion technology has also been introduced into image edge detection, improving detection performance by fusing information from different modalities.

[0003] Traditional operator-based detection methods, while computationally simple and fast, are sensitive to noise, easily leading to false positives and false negatives, and struggle to accurately extract straight line edges in complex backgrounds. The Hough transform method is significantly affected by threshold and Hough spatial resolution, requiring manual parameter adjustment based on the specific image, and cannot detect "thick straight lines" spanning multiple pixels. Deep learning-based detection methods, while robust, suffer from low accuracy in noisy images and challenging conditions, and typically require extensive labeled data for training. Multi-information fusion methods, while improving detection performance to some extent, primarily focus on edge detection after image fusion, lacking targeted optimization for high-precision detection of straight line edges on finite planes. Summary of the Invention

[0004] To address the aforementioned problems, this invention proposes a high-precision algorithm for detecting straight line edges on a finite plane based on the fusion of image and depth information. Its main objectives are: 1) to fully utilize the advantages of both image and depth information to improve the accuracy and robustness of straight line edge detection; 2) to optimize the detection algorithm for the specific characteristics of a finite plane, enabling it to more accurately identify and locate straight line edges, reducing false detections and missed detections; and 3) to reduce reliance on manual parameter adjustment, improving the algorithm's automation and applicability, and enabling it to operate stably in various complex scenarios.

[0005] The first aspect of this invention provides a method for establishing the equation of the edge space of a straight line on a finite plane, comprising the following steps: S1, capture and obtain a finite planar image of a specific scene; S2, based on the depth information of the finite plane and ground stored in the pixel matrix, performs line edge detection of depth information and calculates the standard deviation of the distance between the edge point and the line edge; S3. Using the sampling points selected in the plane inside the straight edge detected by depth information, a finite plane depth equation is established to represent the quantitative relationship between the depth value corresponding to the pixel and the pixel coordinates, and the coefficients of the depth equation are calculated based on the least squares method. S4. Use the Canny algorithm to perform edge detection on the image, obtain the coordinates of the edge pixels, and calculate the standard deviation of the distance between the edge points and the straight edge. S5. If the standard deviation of the straight edge detection in S4 is higher than the standard deviation of the straight edge detection in S2, then the detection result of S2 shall be adopted; if the standard deviation of the straight edge detection in S4 is lower than the standard deviation of the straight edge detection in S2, then the detection result of S4 shall be adopted. S6. Based on the pixel coordinates of the obtained straight edge detection results and the finite plane depth equation, the straight edge spatial equation is established by a straight line fitting method.

[0006] Preferably, the line edge detection in S2 specifically includes: Set a depth gradient threshold A point in the pixel matrix when the depth value changes from Change to The change exceeded When, the depth value is considered to be The pixel is a point on the edge of a straight line.

[0007] Preferably, the calculation of the standard deviation of the distance between the edge point and the line edge specifically includes the following process: S21, determine the coordinates of the pixel points at the edge of the straight line, using the depth gradient threshold. n Coordinates of the edge points ; S22, using the least squares method to fit the edge of a straight line. , The slope of the edge of the line. The intercept; S23, calculate the distance from each edge point to the line. Calculate to the straight line vertical distance ;

[0008] S24, based on vertical distance Calculate the average distance The average distance is the sum of the distances from all points to the line divided by the number of points. S25, based on vertical distance and average distance Calculate the standard deviation The formula for measuring the dispersion of distance distribution is: .

[0009] Preferably, S3 specifically includes: S31, Random Collection There are points, whose coordinates are: ,in Represents the depth value, where Fit a plane equation:

[0010] in, These are the components of the normal vector. It is a constant; S32 employs the least squares method to minimize the sum of the squares of the vertical distances from all sampling points to the plane, thereby constructing a finite plane depth equation to represent the quantitative relationship between the depth value corresponding to a pixel and the pixel coordinates, thus overcoming sensor measurement fluctuations and noise; including: Plane equation standardization:

[0011] in, , , ; Construct the error function: For each point The error is:

[0012] The goal is to minimize the sum of the squares of all errors:

[0013] Solve the least squares problem: right Taking the partial derivatives and setting them to zero, we obtain the following system of linear equations:

[0014] After unfolding, we get:

[0015] Solve the system of equations: Solve using linear algebra methods. Thus, the final finite plane depth equation can be obtained.

[0016] Preferably, the specific steps for detecting straight line edges in an image using the Canny algorithm are as follows: S41 uses a Gaussian filter to smooth the image in order to reduce image noise; S42 calculates the gradient strength and direction at each pixel of the image. The gradient strength is calculated using the Sobel operator, and the edges of the image are detected in the horizontal and vertical directions. S43, based on gradient strength and direction, performs non-maximum suppression, which locally maximizes the gradient strength. In the neighborhood of an edge pixel, only the pixel with the largest gradient strength will be retained. S44 uses two thresholds, a high threshold and a low threshold; pixels above the high threshold are considered strong edges, while pixels below the low threshold are considered non-edges; pixels between these two thresholds are considered edges only if they are connected to strong edge pixels. S45, after dual threshold detection, tracks edge pixels and connects them to form continuous edges by connecting pixels above the low threshold to strong edge pixels; S46, After edge tracking is completed, the edges are refined based on the Canny algorithm to ensure the continuity and accuracy of the edges; S47, When using the Canny algorithm to detect straight edges in an image, calculate the standard deviation of the distance between points on the edge and the straight edge. .

[0017] Preferably, the specific process of S45 is as follows: Set high and low thresholds: Select two thresholds, high threshold... and low threshold High thresholds are used to identify strong edges, while low thresholds are used to identify weak edges. Classify edge pixels: Pixels with gradient magnitude greater than... Pixels with gradient magnitudes less than 1 are marked as strong edges; Pixels are marked as non-edges; gradient magnitude is set in and Pixels between these points are marked as weak edges; Connect weak edges: Examine weak edge pixels. If a weak edge pixel is connected to a strong edge pixel (i.e., within an 8-neighborhood), retain it as an edge; otherwise, mark it as a non-edge.

[0018] Preferably, in step S6, establishing the equation of the line edge space through line fitting involves using the least squares method to fit the equation of the line edge space; the specific process includes: Let the coordinates of any point on the edge of the line be... ,in, These are the pixel coordinates. If the depth value corresponds to the pixel coordinates, then the coordinates in space are:

[0019]

[0020]

[0021] in, These are the camera's parameters; These are the coefficients of the finite plane depth equation; Calculate the centroid: First, determine the centroid of all data points. , where each coordinate is the average value of its components; Decentralized data: each point Convert to coordinates relative to the centroid:

[0022] Construct the covariance matrix: Calculate the covariance matrix Its elements are the covariances of each component:

[0023] Eigenvalue decomposition: For covariance matrix Perform eigenvalue decomposition to obtain eigenvalues. and the corresponding feature vector ; Determine the direction of the line: Maximum eigenvalue corresponding feature vector That is, the direction vector of the fitted line; Establish the equation of the line edge space: The parametric equations for the fitted line edges are:

[0024] in To fit the equation of the straight line edge space, the parameters Represents a real number.

[0025] The second aspect of the present invention provides a method for detecting straight line edges on a finite plane, which uses the straight line edge space equation established by the method described in the first aspect to calculate and obtain the coordinate information of the detected edge and perform corresponding annotation.

[0026] Compared with the prior art, the present invention has the following beneficial effects: This invention combines image and depth information to propose a novel optimized detection algorithm for finite planes. Unlike traditional line detection methods, this algorithm provides high-precision detection of line edges on finite planes, a fusion approach not previously reported in existing technologies. By fusing image and depth information, the proposed algorithm fully leverages the advantages of both, improving the accuracy and robustness of line edge detection and enabling stable operation in various complex scenarios.

[0027] Meanwhile, this invention can effectively improve the accuracy of straight line edge detection on a finite plane, and is applicable to various practical application scenarios such as industrial inspection and autonomous driving, and has broad practical value. Attached Figure Description

[0028] Figure 1 This is a flowchart of the overall process of the present invention.

[0029] Figure 2 This is a schematic diagram of a finite plane.

[0030] Figure 3 This is a schematic diagram of line edge detection based on depth information.

[0031] Figure 4 A schematic diagram is provided for establishing the depth equation for a finite plane.

[0032] Figure 5 The image shown is an RGB image captured in the example.

[0033] Figure 6 This is a depth image captured in the example.

[0034] Figure 7 The example shows the result of straight line edge detection in the depth image (red line).

[0035] Figure 8 The depth equation fitted in this example is shown.

[0036] Figure 9 The example shows line edge detection based on image information (green line, red line is the result of line edge detection based on depth image). Detailed Implementation

[0037] The overall implementation process of this invention is as follows: Figure 1 As shown, the invention will be further described below with reference to specific embodiments.

[0038] I. Description of Finite Plane Test Scenarios The finite plane of this invention refers to a plane in three-dimensional space with straight edges and a finite area, such as... Figure 2As shown. To detect the edges of straight lines, there are two industrial solutions: one is "camera + LiDAR," and the other is "RGBD camera." The straight line edge detection method proposed in this invention is applicable to both solutions, and this invention is specifically designed for detecting straight line edges. A and B The following example illustrates the detailed technical solution.

[0039] II. Line Edge Detection Based on Depth Information by Figure 2 straight edge in B For example, depth information is used for straight-line edge detection. Sensors such as LiDAR or RGBD cameras detect planes with straight edges and finite areas. The depth information of the finite plane and the ground is stored in a pixel matrix. In the pixel matrix, when moving towards the straight edge... B When moving, the depth value will change from Gradually increase to At the edge of the straight line B When near the location, the depth value will change abruptly, such as Figure 3 As shown.

[0040] Set a depth gradient threshold When the depth value is from Change to The change exceeded ,Right now

[0041] The depth value is then considered to be The pixels are straight edges B A little bit above.

[0042] Calculating the standard deviation of the distance between an edge point and the edge of a line involves the following steps: (1) Determine the coordinates of the pixel points at the edge of the straight line, and use the depth gradient threshold to determine n Coordinates of the edge points ; (2) Fit the edge of the line using the least squares method The slope of the edge of the straight line and intercept Calculate using the following formula:

[0043] (3) Calculate the distance from each edge point to the line. Calculate to the straight line vertical distance ;

[0044] (4) Calculate the average distance The average distance is the sum of the distances from all points to the line divided by the number of points. The formula is:

[0045] (5) Calculate the standard deviation Standard deviation is a statistic that measures the dispersion of a distance distribution. The formula for its calculation is:

[0046] III. Establishment of the Depth Equation for a Finite Plane In the second part, although depth information can be used to detect straight edges, external interference factors such as changes in light intensity and occlusion by environmental obstacles may cause deviations between the detected edges and the actual edges. Figure 4 As shown, even with linear fitting of the detected edges, directional and distance deviations will occur. To accurately establish straight edges in space... A and B The equation requires correction of depth error, and this invention proposes to adopt a correction method based on the depth equation.

[0047] A depth equation is a quantitative relationship between the depth value of a pixel and its coordinates. The depth equation for a finite plane in space can be expressed using a plane equation. To describe. This invention selects a suitable number of sampling points inside the edge of a straight line detected using depth information, such as... Figure 4 As shown by the solid points in the diagram, a depth equation is established.

[0048] The specific process is as follows: 1. Sampling point selection Random collection There are points, whose coordinates are: ,in We need to fit a plane equation:

[0049] in, These are the components of the normal vector. It is a constant.

[0050] 2. Least squares fitting The goal of the least squares method is to minimize the sum of the squares of the perpendicular distances from all points to the plane.

[0051] step: (1) Standardization of plane equations: To simplify calculations, the plane equation is usually written as:

[0052] in, , , .

[0053] (2) Construct the error function: For each point The error is:

[0054] The goal is to minimize the sum of the squares of all errors:

[0055] (3) Solving the least squares problem: right Taking the partial derivatives and setting them to zero, we obtain the following system of linear equations:

[0056] After unfolding, we get:

[0057] This is about The system of linear equations can be represented in matrix form as follows:

[0058] (4) Solve the system of equations: Solve using linear algebraic methods (such as Gaussian elimination or matrix inversion). The depth equation can then be obtained. .

[0059] IV. Line edge detection in images using the Canny algorithm The specific steps are as follows: (1) Image preprocessing (a) Grayscale Conversion. Convert the input color image to a grayscale image. This is because the Canny algorithm processes grayscale images. Grayscale conversion can be performed using the following formula:

[0060] in, , , These are the red, green, and blue components of a pixel, respectively.

[0061] (b) Noise Removal. Noise is removed from the grayscale image to reduce interference with edge detection. A Gaussian filter is typically used for smoothing. The kernel size and standard deviation of the Gaussian filter can be adjusted according to the noise level of the image.

[0062] (2) Gradient calculation (a) Calculate the gradient magnitude and direction. Use the Sobel operator to calculate the gradient of the image in the horizontal and vertical directions, respectively. The Sobel operator is defined as follows:

[0063] For each pixel Calculate its horizontal gradient and vertical gradient Then calculate the gradient magnitude. and gradient direction :

[0064] (3) Non-maximum suppression. Identify local maxima by examining the neighboring pixels along the gradient direction for each pixel. If the gradient magnitude of the pixel is a local maximum along its gradient direction, retain the pixel; otherwise, set its gradient magnitude to 0. This step can remove "gaps" at edges and non-edge points.

[0065] (4) Dual threshold detection (a) Set high and low thresholds. Select two thresholds, the high threshold... and low threshold A high threshold is used to identify strong edges, while a low threshold is used to identify weak edges.

[0066] (b) Classify edge pixels: Pixels with gradient magnitude greater than... Pixels with gradient magnitudes less than 1 are marked as strong edges; Pixels are marked as non-edges; gradient magnitude is set in and Pixels between these points are marked as weak edges.

[0067] (c) Connect weak edges: Check weak edge pixels. If they are connected to strong edge pixels (within 8 neighborhoods), keep them as edges; otherwise, mark them as non-edges.

[0068] (5) Calculate the standard deviation of the distance between points on the edge and the straight line edge. Similar to the above steps, the standard deviation corresponding to the image straight line edge detection using the Canny algorithm can be calculated and denoted as . .

[0069] When the standard deviation of straight edge detection The standard deviation of the straight edge detection is higher than that of the standard deviation ... Then, the straight edge detection results based on depth information are used; when the standard deviation of the straight edge detection... The standard deviation of the straight edge detection is lower than that of the standard deviation ... Then, the corresponding pixel coordinates are obtained by using the Canny algorithm to detect straight edges in the image.

[0070] V. Establishment of the equation for the edge space of a straight line Based on the obtained straight edge detection results, i.e. pixel coordinates, a straight edge spatial equation is established using a straight line fitting method.

[0071] Let the coordinates of any point on the edge of the line be... ,in, These are the pixel coordinates. If the depth value corresponds to the pixel coordinates, then the coordinates in space are:

[0072]

[0073]

[0074] in, These are the camera's parameters.

[0075] In three-dimensional space, the least squares method is used to fit the equation of the edge space of a straight line. The specific steps are as follows: 1. Calculate the centroid First, determine the centroid of all data points. The coordinates represent the average values ​​of the components:

[0076] 2. Decentralized data each point Convert to coordinates relative to the centroid:

[0077] 3. Construct the covariance matrix Calculate the covariance matrix Its elements are the covariances of each component:

[0078] 4. Eigenvalue decomposition For covariance matrix Perform eigenvalue decomposition to obtain eigenvalues. and the corresponding feature vector .

[0079] 5. Determine the direction of the straight line. Maximum eigenvalue corresponding feature vector That is, the direction vector of the fitted line.

[0080] 6. Establish the equations for the boundary space of the straight line. The parametric equations for the fitted line edges are:

[0081] in To fit the equation of the straight line edge space, the parameters Represents a real number.

[0082] VI. Experimental Cases This embodiment uses an RGBD camera to photograph a ping-pong table and identifies the left and right edges. The captured RGB image and depth map are shown below. Figure 5 and Figure 6 As shown. If only depth information is used to detect the edges of the ping-pong table, the detection result is as follows. Figure 7 As shown, the detection performance of the left edge (line A) is good, but the detection performance of the right edge (line B) is not good, with a large error compared to the actual edge. After fusing depth information and image information, the fitted depth equation is as follows: Figure 8 and Figure 9 As shown, the detection of the left and right edges (line C and line D) is quite good, and the degree of matching with the actual edges is very high.

[0083] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

[0084] While the specific embodiments of the present invention have been described above, they are not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A method for establishing the equation of the edge space of a straight line on a finite plane, characterized in that, Includes the following processes: S1, capture and obtain a finite planar image of a specific scene; S2, based on the depth information of the finite plane and ground stored in the pixel matrix, performs line edge detection of depth information and calculates the standard deviation of the distance between the edge point and the line edge; S3. Using the sampling points selected in the plane inside the straight edge detected by depth information, a finite plane depth equation is established to represent the quantitative relationship between the depth value corresponding to the pixel and the pixel coordinates, and the coefficients of the depth equation are calculated based on the least squares method. S4. Use the Canny algorithm to perform edge detection on the image, obtain the coordinates of the edge pixels, and calculate the standard deviation of the distance between the edge points and the straight edge. S5. If the standard deviation of the straight edge detection in S4 is higher than the standard deviation of the straight edge detection in S2, then the detection result of S2 is adopted. If the standard deviation of the straight edge detection in S4 is lower than the standard deviation of the straight edge detection in S2, then the detection result of S4 is adopted, and the corresponding pixel coordinates of the detection result are obtained. S6. Based on the pixel coordinates of the obtained straight edge detection results and the finite plane depth equation, the straight edge spatial equation is established by a straight line fitting method.

2. The method for establishing the equation of the edge space of a straight line on a finite plane as described in claim 1, characterized in that: The line edge detection in S2 specifically refers to: Set a depth gradient threshold A point in the pixel matrix when the depth value changes from Change to The change exceeded When, the depth value is considered to be The pixel is a point on the edge of a straight line.

3. The method for establishing the equation of the edge space of a straight line on a finite plane as described in claim 2, characterized in that, The calculation of the standard deviation of the distance between the edge point and the straight line edge specifically includes the following process: S21, determine the coordinates of the pixel points at the edge of the straight line, using the depth gradient threshold. n Coordinates of the edge points ; S22, using the least squares method to fit the edge of a straight line. , The slope of the edge of the line. The intercept; S23, calculate the distance from each edge point to the line. Calculate to the straight line vertical distance ; S24, based on vertical distance Calculate the average distance The average distance is the sum of the distances from all points to the line divided by the number of points. S25, based on vertical distance and average distance Calculate the standard deviation The formula for measuring the dispersion of distance distribution is: 。 4. The method for establishing the equation of the edge space of a straight line on a finite plane as described in claim 1, characterized in that: Specifically, S3 is: S31, Random Collection There are points, whose coordinates are: ,in Represents the depth value, where Fit a plane equation: in, These are the components of the normal vector. It is a constant; S32 employs the least squares method to minimize the sum of the squares of the vertical distances from all sampling points to the plane, thereby constructing a finite plane depth equation to represent the quantitative relationship between the depth value corresponding to a pixel and the pixel coordinates, thus overcoming sensor measurement fluctuations and noise; including: Plane equation standardization: in, , , ; Construct the error function: For each point The error is: The goal is to minimize the sum of the squares of all errors: Solve the least squares problem: right Taking the partial derivatives and setting them to zero, we obtain the following system of linear equations: After unfolding, we get: Solve the system of equations: Solve using linear algebra methods. Thus, the final finite plane depth equation can be obtained.

5. The method for establishing the equation of the edge space of a straight line on a finite plane as described in claim 1, characterized in that: The specific steps for detecting straight edges in an image using the Canny algorithm are as follows: S41 uses a Gaussian filter to smooth the image in order to reduce image noise; S42 calculates the gradient strength and direction at each pixel of the image. The gradient strength is calculated using the Sobel operator, and the edges of the image are detected in the horizontal and vertical directions. S43, based on gradient strength and direction, performs non-maximum suppression, which locally maximizes the gradient strength. In the neighborhood of an edge pixel, only the pixel with the largest gradient strength will be retained. S44 uses two thresholds, a high threshold and a low threshold; pixels above the high threshold are considered strong edges, while pixels below the low threshold are considered non-edges; pixels between these two thresholds are considered edges only if they are connected to strong edge pixels. S45, after dual threshold detection, tracks edge pixels and connects them to form continuous edges by connecting pixels above the low threshold to strong edge pixels; S46, After edge tracking is completed, the edges are refined based on the Canny algorithm to ensure the continuity and accuracy of the edges; S47, When using the Canny algorithm to detect straight edges in an image, calculate the standard deviation of the distance between points on the edge and the straight edge. .

6. The method for establishing the equation of the edge space of a straight line on a finite plane as described in claim 5, characterized in that: The specific process of S45 is as follows: Set high and low thresholds: Select two thresholds, high threshold... and low threshold High thresholds are used to identify strong edges, while low thresholds are used to identify weak edges. Classify edge pixels: Pixels with gradient magnitude greater than 100% Pixels with gradient magnitudes less than 1 are marked as strong edges; Pixels are marked as non-edges; gradient magnitude is set in and Pixels between these points are marked as weak edges; Connect weak edges: Examine weak edge pixels. If a weak edge pixel is connected to a strong edge pixel (i.e., within an 8-neighborhood), retain it as an edge; otherwise, mark it as a non-edge.

7. The method for establishing the equation of the edge space of a straight line on a finite plane as described in claim 1, characterized in that: In S6, the equation of the edge space of a straight line is established by a straight line fitting method, which is to use the least squares method to fit the equation of the edge space of a straight line. The specific process includes: Let the coordinates of any point on the edge of the line be... ,in, These are the pixel coordinates. If the depth value corresponds to the pixel coordinates, then the coordinates in space are: in, These are the camera's parameters; These are the coefficients of the finite plane depth equation; Calculate the centroid: First, determine the centroid of all data points. , where each coordinate is the average value of its components; Decentralized data: each point Convert to coordinates relative to the centroid: Construct the covariance matrix: Calculate the covariance matrix Its elements are the covariances of each component: Eigenvalue decomposition: For covariance matrix Perform eigenvalue decomposition to obtain eigenvalues. and the corresponding feature vector ; Determine the direction of the line: Maximum eigenvalue corresponding feature vector That is, the direction vector of the fitted line; Establish the equation of the line edge space: The parametric equations for the fitted line edges are: in To fit the equation of the straight line edge space, the parameters Represents a real number.

8. A method for detecting the edge of a straight line on a finite plane, characterized in that: Using the linear edge space equation established by the method described in any one of claims 1 to 7, the coordinate information of the detected edge is calculated and labeled accordingly.

Images