Stereoscopic vision boundary feature point extraction method and system

Abstract

The invention discloses a stereoscopic vision boundary feature point extraction method and system, and relates to the field of machine vision and image processing, and the method comprises the steps: deploying two precisely calibrated cameras, and capturing environment images in parallel; preprocessing the captured environment image, and performing parallax optimization and correction on the preprocessed image data; constructing a three-dimensional structure diagram through depth calculation and three-dimensional point cloud conversion, and performing edge extraction and feature point matching on the three-dimensional structure diagram; and finally, the length of the to-be-measured object is obtained through length calculation. By adopting a dual-camera system and utilizing a stereoscopic vision technology, richer spatial information can be obtained, and the extraction accuracy of boundaries and feature points is remarkably improved. The method can effectively solve the problem that a single-view-angle image is limited by view angles and illumination conditions. In addition, the cost aggregation path is dynamically adjusted according to texture characteristics, and the processing performance under different texture conditions is improved.

Description

Technical Field

[0001] The invention relates to the field of machine vision and image processing, and in particular to a method and system for extracting boundary feature points of stereoscopic vision. Background Art

[0002] Internal inspection and maintenance of generators is an important innovative scientific research direction to ensure the normal operation of generators and extend their service life. Traditional inspection methods usually require the removal of some components of the generator, which is not only time-consuming and labor-intensive, but also poses certain safety risks. In recent years, with the development of intelligent and automated technologies, the use of robots for internal inspection of generators has become an important means. Shanghai Electric and Mitsubishi of Japan both have a certain depth of patents. Our invention aims to develop an inspection robot that can walk on the surface of the stator and rotor of the generator. This robot will be equipped with multiple high-definition cameras and other sensors, and through advanced image processing and artificial intelligence technology, it can achieve accurate detection and data analysis of tiny defects inside the generator. Ensure that relevant data can be captured dynamically during the walking process, and then be able to judge the size and size, providing the most basic data content for subsequent neural network calculations.

[0003] In the field of machine vision and automation, accurate extraction of boundaries and feature points is crucial for object recognition and spatial positioning. Traditional methods rely on images from a single perspective, which are often limited by viewing angles and lighting conditions. The present invention uses stereo vision technology to obtain richer spatial information through a dual-camera system, significantly improving the accuracy of feature extraction. Summary of the invention

[0004] In view of the above-mentioned problems, the present invention is proposed.

[0005] Therefore, the problem to be solved by the present invention is that the traditional inspection method usually requires the removal of some components of the generator, which is not only time-consuming and labor-intensive, but also poses certain safety risks; the traditional method relies on images from a single perspective and is often limited by perspective and lighting conditions.

[0006] To solve the above technical problems, the present invention provides the following technical solutions: a method for extracting boundary feature points of stereoscopic vision, which includes deploying two precisely calibrated cameras to capture environmental images in parallel; preprocessing the captured environmental images, and performing parallax optimization and correction on the preprocessed image data; constructing a three-dimensional structure graph through depth calculation and three-dimensional point cloud conversion, and performing edge extraction and feature point matching on the three-dimensional structure graph; and finally obtaining the length of the object to be measured through length calculation.

[0007] As a preferred solution of the stereoscopic vision boundary feature point extraction method described in the present invention, wherein: the environmental image includes images captured by left and right cameras, the image captured by the left camera is recorded as the left image, and the image captured by the right camera is recorded as the right image; the preprocessing includes normalizing and denoising the captured environmental image.

[0008] As a preferred solution of the method for extracting boundary feature points of stereoscopic vision described in the present invention, the disparity optimization and correction includes: performing cost calculation based on a stereo matching algorithm, for each pixel point (x, y) in the left image, searching for the corresponding pixel point (xd, y) in the right image, where d is the range of disparity, which is between 0 and 64 pixels; defining 8 directions, for each direction, calculating the similarity between adjacent pixels, and using the absolute value of the pixel intensity difference as a measure of similarity;

[0009] For pixel p at disparity d, set the initial matching cost C(p,d) and calculate the cumulative cost C for each disparity d of each pixel p along each direction i agg (p,d), the formula is expressed as,

[0010]

[0011] Among them, Δp i represents the pixel offset in direction i, P1 and P2 are represented as penalty terms to balance disparity continuity and pixel intensity difference, and k is represented as; the optimal disparity decision includes, for each pixel p, traversing all possible disparity values ​​d, and selecting the one that makes the cumulative cost C agg The minimum disparity value of (p, d) is expressed as the final disparity of the pixel:

[0012]

[0013] As a preferred solution of the method for extracting boundary feature points of stereoscopic vision described in the present invention, the disparity optimization and correction also includes, for each pixel point in the left image and the disparity calculated therefrom, searching for the corresponding pixel point and disparity in the right image; if the disparity estimates in the left image and the right image are inconsistent, it means that the current disparity estimate is inaccurate; for the inconsistent disparity estimates, mark them as invalid or use the disparity of adjacent pixels for interpolation; for the disparity value of each pixel point, refine it with sub-pixel precision, calculate the vertex position by fitting a quadratic curve to the cumulative cost function near the minimum cost disparity, so as to obtain an accurate disparity value; if d min is the error with the lowest cost, then C(d min -1)、C(d min )、C(d min +1) is the cost of the adjacent disparity value, then the sub-pixel disparity d sub It is expressed as,

[0014]

[0015] The cost aggregation path is dynamically adjusted based on image features, and eight-directional paths are used for cost aggregation in high-texture areas, while the number of paths is reduced in low-texture areas; the texture characteristics of the area are judged by analyzing the local standard deviation or gradient strength of the pixels, and the number of paths is increased in areas with high texture or high gradient, and vice versa; the high texture and low texture judgment method includes setting a threshold T to judge the texture density of the area, if the texture density of an area is greater than T, eight-directional paths are selected for the high-texture area; if it is less than T, four-directional paths are selected for the low-texture area; the texture density σ p The calculation formula is expressed as:

[0016]

[0017] Among them, I(i) represents the intensity of the pixel in the neighborhood, It is represented as the average value of the pixel intensity in the neighborhood, and N is represented as the number of pixels in the neighborhood; the depth map is finally produced after parallax optimization and correction.

[0018] As a preferred solution of the method for extracting boundary feature points of stereoscopic vision described in the present invention, the depth calculation includes, based on the baseline distance B and focal length f of the camera, for a pixel point with parallax d, its depth Z is calculated according to the formula, expressed as:

[0019]

[0020] The generated depth map is smoothed by applying a bilateral filter and a median filter; the three-dimensional point cloud conversion includes converting each pixel in the depth map into a point in three-dimensional space based on the intrinsic and extrinsic parameters of the camera. For each pixel in the depth map and its corresponding depth value, the calculation formula of the position in the world coordinate system is expressed as:

[0021]

[0022]

[0023] Z s =Z

[0024] Among them, (u, v) represents the coordinates of the pixel in the depth map, (cx, cy) represents the optical center position of the camera, and (X, Y, Z s) is expressed as the coordinates of the pixel in the world coordinate system; each point where the pixel falls in the world coordinate system is recorded as a point cloud, and the accuracy optimization of each point cloud includes minimizing the sum of the square distances between the points of two point clouds; the quality control of each point cloud includes calculating the number of neighbors of each point cloud, estimating the density of the point cloud, setting a threshold, and if the number of neighbors of a point cloud is less than the threshold, the point cloud is removed.

[0025] As a preferred solution of the method for extracting boundary feature points of stereoscopic vision described in the present invention, the edge extraction and feature point matching include applying a Gaussian filter to process the position (x, y) of each pixel point in the image relative to the center of the kernel as G(x, y), expressed as:

[0026]

[0027] Where σ is the standard deviation of the Gaussian filter; applying the Gaussian kernel to each pixel using convolution is expressed as,

[0028] I smooth (x, y) = G(x, y) * I(x, y)

[0029] Where I(,) represents the convolution operation; calculate the level G of each pixel in the image x and vertical G y The gradient is expressed as,

[0030]

[0031] Calculate the magnitude of the gradient G mag and direction θ is expressed as,

[0032]

[0033] Go through each pixel and apply threshold T b To identify the edge representation as,

[0034] if G mag (x,y)>T then mark as edge

[0035] The Harris corner response value of each pixel is calculated to identify the key points in the image. For each detected key point, the gradient consistency in multiple directions is calculated, and the key points with consistent gradients in different directions are selected as stable feature points. For each key point, a pixel neighborhood is selected and the neighborhood is divided into several sub-blocks of pixels. The gradient direction and magnitude are calculated for each sub-block. For each sub-block, the gradient histogram is divided into 8 intervals according to the gradient direction, and the sum of the gradient magnitudes in each interval is accumulated. The gradient histograms of all sub-blocks are connected in series to form the feature vector of the key point. The edge pixels in the image are marked based on the feature vector of the key point. With an edge point as the starting point, the connected edge points are traversed using a depth-first search, and the coordinate sequence of each edge point is recorded, which is integrated into a set of coordinate points and recorded as the contour L. The contour L is smoothed to generate a smoothed contour L′, and the coordinate point sequence of L′ is replaced with the original coordinate point sequence of L. Feature point matching is performed based on the similarity measure between vectors in another perspective.

[0036] As a preferred solution of the method for extracting boundary feature points of stereoscopic vision described in the present invention, the length calculation includes, for each viewing angle, calculating the straight-line distance between key points A and B as:

[0037]

[0038]

[0039] The weighted average is used to combine the measurements D and D' of the two views to obtain the final measured length D final It is expressed as,

[0040]

[0041] Among them, W and W' represent the weights of the corresponding perspectives.

[0042] Another object of the present invention is to provide a stereoscopic vision boundary feature point extraction system, which can identify the morphological features of the internal components of the generator.

[0043] To solve the above technical problems, the present invention provides the following technical solutions: a system for a method for extracting boundary feature points of stereoscopic vision, comprising: an image acquisition module, an image data optimization and correction module, a feature extraction and matching module and a length calculation module; the image acquisition module captures environmental images in parallel; the image data optimization and correction module preprocesses the captured environmental images, and performs parallax optimization and correction on the preprocessed image data; the feature extraction and matching module constructs a three-dimensional structure graph through depth calculation and three-dimensional point cloud conversion, and performs edge extraction and feature point matching on the three-dimensional structure graph; the length calculation module finally obtains the length of the object to be measured through length calculation.

[0044] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above-mentioned method for extracting boundary feature points of stereoscopic vision when executing the computer program.

[0045] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the above-mentioned method for extracting boundary feature points of stereoscopic vision.

[0046] The beneficial effects of the present invention are as follows: by adopting a dual camera system and utilizing stereo vision technology, the present invention can obtain richer spatial information and significantly improve the accuracy of extracting boundaries and feature points. This method can effectively deal with the problem that single-view images are limited by viewing angles and lighting conditions.

[0047] The present invention dynamically adjusts the cost aggregation path according to the texture characteristics, improving the processing performance under different texture conditions. Through depth calculation and three-dimensional point cloud conversion, the two-dimensional image data is effectively converted into a three-dimensional structure, providing detailed three-dimensional data support for further analysis. The present invention can also more accurately measure and analyze the length of the object to be measured by combining the measurement results of two perspectives, which is particularly important for precision engineering and quality control. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. Among them:

[0049] Figure 1 This is a flow chart of a method for extracting boundary feature points of stereoscopic vision in Example 1.

[0050] Figure 2 This is a module structure diagram of a system for a method for extracting boundary feature points of stereoscopic vision in Example 3. DETAILED DESCRIPTION

[0051] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the accompanying drawings.

[0052] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0053] Example 1, reference Figure 1 , which is the first embodiment of the present invention, and which provides a method for extracting boundary feature points of a stereoscopic vision, including: Figure 1 As shown:

[0054] S1. Deploy two precisely calibrated cameras to capture environment images in parallel.

[0055] Dual-camera stereo vision system setup:

[0056] Camera type: Two 1080p resolution CMOS cameras.

[0057] Focal length and field of view: 35mm focal length, 90 degree field of view.

[0058] Camera distance: fixed at 65mm.

[0059] Synchronous trigger mechanism: Hardware synchronization is used to ensure that the image capture time of the two cameras is consistent to avoid timing differences.

[0060] The environmental image includes images captured by the left and right cameras. The image captured by the left camera is recorded as the left image, and the image captured by the right camera is recorded as the right image.

[0061] S2. Preprocess the captured environment image, and perform parallax optimization and correction on the preprocessed image data.

[0062] Preprocessing includes normalization: converting the raw image data captured by the camera (usually integer values ​​from 0 to 255) into a floating point range from 0 to 1 to unify the data format. (The purpose of normalization is to preprocess the image data so that the pixel values ​​are within a uniform range, eliminate the influence of external factors such as lighting changes, and enhance the stability and accuracy of the image processing algorithm.)

[0063] Denoising: Apply a Gaussian blur filter with a radius of 5 pixels and a standard deviation of 1.5 to smooth the image and reduce noise.

[0064] Disparity optimization and correction include cost calculation based on the stereo matching algorithm. For each pixel point (x, y) in the left image, the corresponding pixel point (xd, y) is searched in the right image, where d is the range of disparity, which is between 0 and 64 pixels. Eight directions are defined, and for each direction, the similarity between adjacent pixels is calculated, and the absolute value of the pixel intensity difference is used as a measure of similarity.

[0065] For pixel p at disparity d, set the initial matching cost C(p, d) and calculate the cumulative cost C for each disparity d of each p pixel along each direction i agg (p, d), the formula is expressed as,

[0066]

[0067] Among them, Δp i represents the pixel offset in direction i, P1 and P2 are penalty terms used to balance disparity continuity and pixel intensity difference, and k is represented as;

[0068] Making the optimal disparity decision involves, for each pixel p, traversing all possible disparity values ​​d and selecting the one that makes the cumulative cost C agg The minimum disparity value of (p, d) is expressed as the final disparity of the pixel:

[0069]

[0070] Left-right consistency check: For each pixel in the left image and its calculated disparity, find the corresponding pixel and disparity in the right image. If the disparity estimates in the left and right images are inconsistent, it means that the current disparity estimate is inaccurate. For inconsistent disparity estimates, mark them as invalid or use the disparity of adjacent pixels for interpolation.

[0071] Disparity refinement: For the disparity value of each pixel, sub-pixel precision is used for refinement. The vertex position is calculated by fitting a quadratic curve around the minimum cost disparity to the cumulative cost function to obtain an accurate disparity value.

[0072] If d min is the error with the lowest cost, then C(d min -1)、C(d min )、C(d min +1) is the cost of the adjacent disparity value, then the sub-pixel disparity d sub It is expressed as,

[0073]

[0074] The cost aggregation path is dynamically adjusted based on image features. In high-texture areas, eight-directional paths are used for cost aggregation, while the number of paths is reduced in low-texture areas.

[0075] By analyzing the local standard deviation or gradient strength of the pixels, the texture characteristics of the area are determined, and the number of paths is increased in areas with high texture or high gradient, and decreased otherwise.

[0076] The method for judging high texture and low texture includes setting a threshold T to judge the texture density of the area. If the texture density of an area is greater than T, eight directions of paths are selected for the high texture area; if it is less than T, four directions of paths are selected for the low texture area.

[0077] Texture density σ pThe calculation formula is expressed as:

[0078]

[0079] Among them, I(i) represents the intensity of the pixel in the neighborhood, It is represented as the average value of pixel intensity in the neighborhood, and N is the number of pixels in the neighborhood.

[0080] Through the application of these optimization and innovative strategies, it is possible to improve the accuracy of disparity estimation while optimizing the computational efficiency of the algorithm and adapting to image areas with different texture characteristics, thereby generating more accurate and high-quality depth maps.

[0081] S3. Construct a three-dimensional structure graph through depth calculation and three-dimensional point cloud conversion, and perform edge extraction and feature point matching on the three-dimensional structure graph.

[0082] Based on the baseline distance B and focal length f of the camera, for a pixel with parallax d, its depth Z is calculated according to the formula, expressed as,

[0083]

[0084] For example, the baseline distance B of the camera is 65mm (0.065m) and the focal length f is 35mm.

[0085] For a pixel with disparity d, its depth Z can be calculated according to the formula:

[0086]

[0087] If the disparity d of a pixel is 0.005m, then the depth Z of the point is:

[0088]

[0089] Noise filtering and correction: Bilateral filter and median filter are applied to the generated depth map for smoothing.

[0090] Bilateral filtering: Select a bilateral filter with a diameter of 9, set the standard deviation of the color space to 75, and the standard deviation of the space to 75 to remove noise while maintaining edges.

[0091] Median filtering: Use a 5x5 window for median filtering to further remove outliers.

[0092] 3D point cloud coordinate conversion: Each pixel in the depth map needs to be converted into a point in 3D space. This process depends on the intrinsic and extrinsic parameters of the camera.

[0093] Intrinsic parameters: For example, the focal length f and the optical center position (cx, cy). Intrinsic parameters are used to transform pixel coordinates into points in the camera coordinate system.

[0094] Extrinsic parameters: Describe the position and orientation of the camera in the world coordinate system. Extrinsic parameters are used to convert points in the camera coordinate system to points in the world coordinate system.

[0095] Coordinate conversion formula: For each pixel in the depth map and its corresponding depth value, the calculation formula for the position in the world coordinate system is expressed as,

[0096]

[0097]

[0098] Z s =Z

[0099] Where (u, v) represents the coordinates of the pixel in the depth map, (X, Y, Z s ) is expressed as the pixel's coordinates in the world coordinate system.

[0100] Point cloud accuracy optimization includes: goal: minimizing the sum of square distances between points in two point clouds (eg, point clouds obtained from left and right cameras).

[0101] Process: Iteratively changes the position and orientation of a point cloud to achieve the best alignment. In each iteration, the distances between corresponding points are calculated and the position of the point cloud is adjusted to reduce these distances.

[0102] End condition: Stop when a certain number of iterations is reached or the alignment error is lower than a certain threshold.

[0103] Point cloud density and quality control include,density analysis: calculating the number of neighbors of each point in the point cloud,which is used to estimate the density of the point cloud.

[0104] Quality control includes,removing isolated points: if the number of neighbors of a point is less than a certain threshold, it is considered,isolated, possibly noise, and should be removed.

[0105] Outlier detection: Identify and remove outliers by analyzing the statistical distribution of the point cloud, such as using the mean and standard deviation.

[0106] Edge extraction and feature point matching: For each pixel in the image relative to the position of the kernel center (x, y), a Gaussian filter is applied to process it into G(x, y), which is expressed as,

[0107]

[0108] Where σ is the standard deviation of the Gaussian filter; applying the Gaussian kernel to each pixel using convolution is expressed as,

[0109] I smooth(x, y) = G(x, y) * I(x, y)

[0110] Among them, I(,) represents the convolution operation.

[0111] Calculate the level G of each pixel in the image x and vertical G y The gradient is expressed as,

[0112]

[0113] Calculate the magnitude of the gradient G mag and direction θ is expressed as,

[0114]

[0115] Go through each pixel and apply threshold T b To identify the edge representation as,

[0116] if G mag (x,y)>T then mark as edge

[0117] Key point detection: Identify key points in an image, which are points with the largest gradient change in their surrounding neighborhood. Calculate the Harris corner response value R of each pixel p(x, y) to identify key points in the image. The formula is:

[0118] R=λ1λ2-k(λ1+λ2) 2

[0119] Among them, λ1, λ2 represent the eigenvalues ​​of the local autocorrelation matrix of the pixel point, and k represents an empirical constant.

[0120] Keypoint stability evaluation: For each detected keypoint, calculate its gradient consistency in multiple directions. Select points that show high gradient consistency in different directions as stable feature points.

[0121] Data processing to build descriptors:

[0122] Determine neighborhood size: For each keypoint, select a neighborhood of 16x16 pixels.

[0123] Sub-block partitioning: The neighborhood is divided into sub-blocks of 4x4 pixels, with a total of 16 sub-blocks.

[0124] Gradient calculation: For each sub-block, the Sobel operator is used to calculate the gradient direction and magnitude. For example, for pixel point (i, j), the gradient direction θ(i, j) and magnitude G(i, j) are calculated as follows:

[0125]

[0126]

[0127] Gradient histogram: For each sub-block, the gradient histogram is divided into 8 intervals according to the gradient direction, and the sum of the gradient magnitudes in each interval is accumulated.

[0128] Feature vector construction: The gradient histograms of the 16 sub-blocks are concatenated to form a 128-dimensional feature vector.

[0129] Data processing for contour tracing:

[0130] Edge detection: Marking edge pixels in an image.

[0131] Tracking algorithm: Take an edge point as the starting point and use depth-first search to traverse the connected edge points.

[0132] Contour data recording: Record the coordinate sequence of each edge point. The contour L is represented as a set of coordinate points {P1, P2, ..., Pn}.

[0133] Smoothing algorithm: Use polynomial or spline curve fitting to smooth the contour lines. For the above contour L, apply the spline curve fitting algorithm to generate a smoothed contour L'.

[0134] Data conversion: Convert the original coordinate point sequence into a smoothed coordinate point sequence {P′1, P′2, ..., P′n}.

[0135] Data format display, the following are two different data format displays:

[0136] 1. Data format of feature descriptor:

[0137] Displayed in array or matrix form. For example, a 128-dimensional feature vector can be represented as a 1x128 array.

[0138] Each element represents the sum of the gradients in a certain gradient direction interval of a sub-block.

[0139] 2. Format of profile data:

[0140] Original contour: displayed as a sequence of coordinate points, such as {(x1, y1), (x2, y2), ..., (xn, yn)}.

[0141] Smooth contour: displayed as a sequence of adjusted coordinate points, such as {(x′1, y′1), (x′2, y′2), ..., (x′n, y′n)}.

[0142] The final data is as follows:

[0143] 3. Feature descriptor data:

[0144] The feature vector of key point A: A =[v A1 , v A2 , ..., v A128 ]]

[0145] The feature vector of key point B: B =[v B1 , v B2 , ..., v B128 ]]

[0146] 4. Contour data:

[0147] Coordinates of key point A (at one viewing angle): [P A =(X A ,y A , z A )]

[0148] Coordinates of key point B (at the same viewing angle): [P B =(x B ,y B , z B )]

[0149] Data calculation and deduction, feature point matching:

[0150] In another perspective, we look for the same A ) and (\textbfV B )The most matching feature point. Assume that the coordinates of the matched feature points are (P′ A =(x′ A , y′ A , z′ A )) and (P′ B =(x′ B , y′ B , z′ B )).

[0151] Feature point matching can be performed based on the Euclidean distance or other similarity measures between vectors.

[0152] S4. Finally, the length of the object to be measured is obtained by length calculation.

[0153] For each viewpoint, the straight-line distance between key points A and B is calculated as,

[0154]

[0155]

[0156] The weighted average is used to combine the measurements D and D' of the two views to obtain the final measured length D final It is expressed as,

[0157]

[0158] Among them, W and W' represent the weights of the corresponding perspectives.

[0159] Embodiment 2 is the second embodiment of the present invention, which is different from the first embodiment in that: a method for extracting boundary feature points of stereoscopic vision also includes: in order to verify and illustrate the technical effects adopted in this method, this embodiment adopts a traditional technical solution and the method of the present invention for comparative testing, and compares the test results by means of scientific argumentation to verify the real effect of this method.

[0160] Experimental design

[0161] Experimental environment: The internal image data of the generator in a standardized industrial environment was selected as the test object.

[0162] Experimental object: Using the same test image set, the method of the present invention is compared with the prior art solution (an image processing method that relies on a single viewing angle).

[0163] Test indicators: The following four core performance indicators are selected for evaluation:

[0164] Edge detection accuracy: evaluates the accuracy of the method in identifying image edges.

[0165] Feature point positioning accuracy: measures the accuracy of the method in determining the location of image feature points (such as corner points, edge intersections, etc.).

[0166] Processing time: The total time required to calculate from the input image to the output result, in seconds.

[0167] Disparity estimation error: Measures the average error in estimating the disparity of an image, where smaller values ​​indicate more accurate estimates.

[0168] The experimental data obtained are shown in Table 1.

[0169] Table 1: Experimental data comparison table

[0170]

[0171]

[0172] Edge detection accuracy: The edge detection accuracy of the method of the present invention is improved by 15% compared with the prior art solution, which shows that through stereo vision technology, the present invention can more accurately identify the edges in the image, thereby improving the accuracy of the overall image analysis.

[0173] Feature point positioning accuracy: The method of the present invention provides higher feature point positioning accuracy (93%), which is 8% higher than the existing technology, which is crucial for object recognition and spatial positioning in complex environments, and is particularly valuable in automation and robotics.

[0174] Processing time: In terms of processing time, the method of the present invention is 5 seconds faster than the prior art solution, which shows that the algorithm of the present invention has obvious advantages in efficiency and is suitable for real-time processing and analysis of large amounts of image data.

[0175] Disparity estimation error: The disparity estimation error is reduced from 0.03 to 0.01, indicating that the method of the present invention is more accurate in calculating and correcting disparity, which directly affects the quality of the depth map and the accuracy of 3D reconstruction.

[0176] Example 3, reference Figure 2 , which is the third embodiment of the present invention, and is different from the previous two embodiments in that: a system for a method for extracting boundary feature points of stereoscopic vision includes an image acquisition module 100, an image data optimization and correction module 200, a feature extraction and matching module 300 and a length calculation module 400; the image acquisition module 100 captures environmental images in parallel; the image data optimization and correction module 200 preprocesses the captured environmental images, and performs parallax optimization and correction on the preprocessed image data; the feature extraction and matching module 300 constructs a three-dimensional structure graph through depth calculation and three-dimensional point cloud conversion, and performs edge extraction and feature point matching on the three-dimensional structure graph; the length calculation module 400 finally obtains the length of the object to be measured through length calculation.

[0177] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program codes.

[0178] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0179] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.

[0180] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0181] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A method for extracting boundary feature points of stereoscopic vision, characterized in that: include, Deploy two precisely calibrated cameras to capture environment images in parallel; Preprocess the captured environment image, and perform parallax optimization and correction on the preprocessed image data; Construct a 3D structure graph through depth calculation and 3D point cloud conversion, and perform edge extraction and feature point matching on the 3D structure graph; Finally, the length of the object to be measured is obtained through length calculation.

2. A method for extracting boundary feature points of stereoscopic vision according to claim 1, characterized in that: The environmental image includes images captured by left and right cameras, the image captured by the left camera is recorded as the left image, and the image captured by the right camera is recorded as the right image; The preprocessing includes normalizing and denoising the captured environment image.

3. A method for extracting boundary feature points of stereoscopic vision as claimed in claim 2, characterized in that: The disparity optimization and correction includes: performing cost calculation based on a stereo matching algorithm, searching for a corresponding pixel point (xd, y) in the right image for each pixel point (x, y) in the left image, where d is the range of the disparity, which is between 0 and 64 pixels; defining 8 directions, and for each direction, calculating the similarity between adjacent pixels, using the absolute value of the pixel intensity difference as a measure of similarity; For pixel p at disparity d, set the initial matching cost C(p,d) and calculate the cumulative cost C for each disparity d of each pixel p along each direction i agg (p,d), the formula is expressed as, Among them, Δp i represents the pixel offset in direction i, P1 and P2 are penalty terms used to balance disparity continuity and pixel intensity difference, and k is represented as; Making the optimal disparity decision involves, for each pixel p, traversing all possible disparity values ​​d and selecting the one that makes the cumulative cost C agg The minimum disparity value of (p, d) is expressed as the final disparity of the pixel:

4. A method for extracting boundary feature points of stereoscopic vision as claimed in claim 3, characterized in that: The disparity optimization and correction also includes, for each pixel point in the left image and the disparity calculated therefrom, searching for the corresponding pixel point and disparity in the right image, if the disparity estimates in the left image and the right image are inconsistent, it means that the current disparity estimate is inaccurate, and for the inconsistent disparity estimates, marking them as invalid or interpolating using the disparity of adjacent pixels; For the disparity value of each pixel, sub-pixel precision is used for refinement, and the vertex position is calculated by fitting a quadratic curve around the minimum cost disparity of the cumulative cost function to obtain an accurate disparity value; If d min is the error with the lowest cost, then C(d min -1)、C(d min )、C(d min +1) is the cost of the adjacent disparity value, then the sub-pixel disparity d sub It is expressed as, Dynamically adjust the cost aggregation path based on image features, use 8-directional paths for cost aggregation in high-texture areas, and reduce the number of paths in low-texture areas; By analyzing the local standard deviation or gradient strength of pixels, the texture characteristics of the area are determined, and the number of paths is increased in areas with high texture or high gradient, and decreased in areas with low texture or high gradient. The method for determining high texture and low texture includes setting a threshold T to determine the texture density of a region, and if the texture density of a region is greater than T, selecting eight directions of paths for the high texture region; if it is less than T, selecting four directions of paths for the low texture region; Texture density σ p The calculation formula is expressed as: Among them, I(i) represents the intensity of the pixel in the neighborhood, It is represented as the average value of pixel intensity in the neighborhood, and N is represented as the number of pixels in the neighborhood; The final depth map is produced after parallax optimization and correction.

5. A method for extracting boundary feature points of stereoscopic vision as claimed in claim 4, characterized in that: The depth calculation includes, based on the baseline distance B and focal length f of the camera, for a pixel point with parallax d, its depth Z is calculated according to the formula, expressed as: Apply bilateral filter and median filter to smooth the generated depth map; The three-dimensional point cloud conversion includes converting each pixel in the depth map into a point in three-dimensional space based on the intrinsic and extrinsic parameters of the camera. For each pixel in the depth map and its corresponding depth value, the calculation formula of the position in the world coordinate system is expressed as: WITH s =Z Among them, (u,v) represents the coordinates of the pixel in the depth map, (cx,cy) represents the optical center position of the camera, and (X,Y,Z s ) is expressed as the coordinates of the pixel in the world coordinate system; The point where each pixel falls in the world coordinate system is recorded as a point cloud. The accuracy optimization of each point cloud includes minimizing the sum of the square distances between the points of two point clouds; The quality control of each point cloud includes calculating the number of neighbors of each point cloud, estimating the density of the point cloud, setting a threshold, and removing the point cloud if the number of neighbors of a point cloud is less than the threshold.

6. A method for extracting boundary feature points of stereoscopic vision according to claim 5, characterized in that: The edge extraction and feature point matching include applying a Gaussian filter to the position (x, y) of each pixel in the image relative to the center of the kernel to process it into G(x, y), which is expressed as: Where σ represents the standard deviation of the Gaussian filter; Using convolution to apply a Gaussian kernel to each pixel is represented as, I smooth (x,y)=G(x,y)*I(x,y) Among them, I(,) represents the convolution operation; Calculate the level G of each pixel in the image x and vertical G y The gradient is expressed as, Calculate the magnitude of the gradient G mag and direction θ is expressed as, Go through each pixel and apply threshold T b To identify the edge representation as, if G mag (x,y)>T then mark as edge Calculate the Harris corner response value of each pixel to identify the key points in the image. For each detected key point, calculate the gradient consistency in multiple directions and select the key point with consistent gradients in different directions as a stable feature point. For each key point, select a pixel neighborhood and divide the neighborhood into several pixel sub-blocks. Calculate the gradient direction and magnitude for each sub-block. For each sub-block, divide the gradient histogram into 8 intervals according to the gradient direction, and accumulate the sum of the gradient magnitudes in each interval. Concatenate the gradient histograms of all sub-blocks to form the feature vector of the key point. Mark edge pixels in the image based on the feature vector of the key point, take an edge point as the starting point, use depth-first search to traverse the connected edge points, record the coordinate sequence of each edge point, and integrate them into a set of coordinate points recorded as contour L; smooth the contour L to generate a smoothed contour L', and replace the original L coordinate point sequence with the coordinate point sequence of L'; In another perspective, feature point matching is performed based on the similarity measure between vectors.

7. A method for extracting boundary feature points of stereoscopic vision according to claim 6, characterized in that: The length calculation includes, for each viewing angle, calculating the straight-line distance between key points A and B as: The weighted average is used to combine the measurements D and D' of the two views to obtain the final measured length D final It is expressed as, Among them, W and W' represent the weights of the corresponding perspectives.

8. A system using a method for extracting boundary feature points of stereoscopic vision as claimed in any one of claims 1 to 7, characterized in that: It comprises an image acquisition module (100), an image data optimization and correction module (200), a feature extraction and matching module (300) and a length calculation module (400); The image acquisition module (100) captures environmental images in parallel; The image data optimization and correction module (200) pre-processes the captured environment image, and performs parallax optimization and correction on the pre-processed image data; The feature extraction and matching module (300) constructs a three-dimensional structure graph through depth calculation and three-dimensional point cloud conversion, and performs edge extraction and feature point matching on the three-dimensional structure graph; The length calculation module (400) finally obtains the length of the object to be measured through length calculation.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of a method for extracting boundary feature points of stereoscopic vision described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of a method for extracting boundary feature points of stereoscopic vision described in any one of claims 1 to 7 are implemented.

Images