Binocular vision space tracking and positioning method and system based on improved SIFT (Scale Invariant Feature Transform) feature points

Through feature point extraction and adaptive window disparity calculation of Hessian matrix and Gaussian differential DoG pyramid, combined with the energy function optimization of graph cutting algorithm, the problems of low parallax calculation accuracy and sensitivity to light and viewing angle changes in the existing technology are solved, and high Accurate feature points 3D reconstruction and camera motion tracking.

CN119068162APending Publication Date: 2024-12-03GUIZHOU POWER GRID CO LTD
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202410924993.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-11
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

The binocular visual spatial tracking and positioning method based on SIFT feature points in the prior art is inefficient in large-scale scenes and complex backgrounds, is sensitive to illumination and viewing angle changes, and is unable to adapt to local texture changes in the image, resulting in parallax calculation accuracy and feature points. Inadequate extraction accuracy.

Method used

The Hessian matrix and Gaussian differential DoG pyramid are used for feature points extraction and matching, and the parallax calculation method and graph cutting algorithm are introduced to optimize the energy function of the adaptive window, and the triangulation calculation is used for the disparity value and camera baseline to realize the local texture and depth of the image. Adaptation to changes improves the reconstruction accuracy of three-dimensional coordinates of feature points.

Benefits of technology

It improves the accuracy and robustness of parallax calculation, realizes high-precision extraction and matching of feature points, provides accurate three-dimensional spatial information for camera motion tracking, and enhances the adaptability and real-time nature of the algorithm.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119068162A_ABST
    Figure CN119068162A_ABST
Patent Text Reader

Abstract

The invention discloses a binocular vision space tracking and positioning method and system based on improved SIFT feature points, and relates to the technical field of computer vision, and the method comprises the steps: collecting and preprocessing image data and camera parameters, and carrying out the feature point extraction and feature point matching of the image data; calculating parallax values of the feature points according to matching results of the feature points; calculating the position of a feature point by using the parallax value and the camera parameter to obtain a three-dimensional coordinate of the feature point; performing motion tracking based on the three-dimensional coordinates to obtain the state of the camera; and optimizing the state of the camera to realize tracking and positioning of the target object. According to the method, feature point extraction and matching are carried out by using an SURF algorithm and a multi-scale matching method, so that accurate extraction and high-quality matching of features are realized; a parallax calculation method of an adaptive window is introduced, so that the accuracy and robustness of a parallax value are improved; and the reprojection error is optimized by using the least square method, and the accuracy of the three-dimensional coordinates of the feature points is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to a binocular vision spatial tracking and localization method and system based on improved SIFT feature points. Background Technology

[0002] Visual tracking and localization technology has wide applications in robot navigation, augmented reality, and drone aerial photography. With the rapid development of computer vision technology, image processing-based visual tracking and localization methods have gradually become a research hotspot. Among them, the binocular visual spatial tracking and localization algorithm based on SIFT feature points is a classic visual localization method. By combining feature point extraction, matching, and disparity calculation techniques, it uses two cameras to acquire images, calculates the disparity values ​​of feature points in the images, and reconstructs three-dimensional spatial coordinates, thereby achieving accurate positioning and motion tracking of the target object. It has the advantages of scale invariance and rotation invariance and has wide applications in image processing and computer vision. In recent years, although deep learning has made significant breakthroughs in the field of computer vision, especially in... While SIFT performs exceptionally well in object detection and image classification tasks, traditional handcrafted feature-based visual localization methods still hold advantages. For instance, deep learning-based visual localization methods require a large amount of labeled data for training and have high hardware resource requirements, SLAM-based visual localization requires the construction of complex environmental maps, and localization methods based on visual markers and structured light are also subject to certain scene limitations. In contrast, the binocular visual localization algorithm based on SIFT feature points has the advantages of requiring no training data, having low hardware requirements, and being widely applicable in small-scale scenes and with limited computing resources. Therefore, applying the SIFT algorithm to binocular visual spatial tracking and localization can improve the detection and matching accuracy of feature points, providing a reliable foundation for 3D reconstruction and motion tracking.

[0003] However, current common solutions have many drawbacks, including: traditional disparity calculation methods usually use a fixed window size, which cannot adapt to local texture changes in the image, resulting in low disparity calculation accuracy; the SIFT algorithm has low computational efficiency in large-scale scenes and complex backgrounds, resulting in insufficient real-time performance; the SIFT algorithm is sensitive to changes in lighting and viewpoint, affecting the accuracy of feature point extraction and matching; and the SIFT algorithm may not be able to extract effective feature points in areas lacking obvious texture or repetitive structures. Summary of the Invention

[0004] Given that existing technologies for visual tracking and localization of target objects typically use a fixed window size, which cannot adapt to local texture changes in images, resulting in low parallax calculation accuracy, low computational efficiency in large-scale scenes and complex backgrounds, sensitivity to changes in illumination and viewpoint, affecting the accuracy of feature point extraction and matching, and low accuracy in feature point extraction in areas lacking obvious textures or repetitive structures, this invention is proposed.

[0005] Therefore, the problem to be solved by this invention is to provide a method for accurately extracting and matching features in an image using the Hessian matrix and the Difference of Gaussian (DoG) pyramid, introducing an adaptive window disparity calculation method and a graph cut algorithm energy function optimization method to adapt to local texture and depth changes in the image, and using disparity values ​​and camera baselines for triangulation calculation to achieve high-precision reconstruction of the three-dimensional coordinates of feature points.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0007] In a first aspect, embodiments of the present invention provide a binocular visual spatial tracking and positioning method based on improved SIFT feature points, which includes acquiring image data and camera parameters and performing preprocessing; extracting and matching feature points in the image data; calculating the disparity value of the feature points based on the matching results; calculating the position of the feature points using the disparity value and the camera parameters to obtain the three-dimensional coordinates of the feature points; performing motion tracking based on the three-dimensional coordinates to obtain the state of the camera; and optimizing the state of the camera to achieve tracking and positioning of the target object.

[0008] As a preferred embodiment of the binocular visual spatial tracking and localization method based on improved SIFT feature points described in this invention, the image data includes images from the left camera and the right camera; the camera parameter is the baseline length of the camera.

[0009] As a preferred embodiment of the binocular visual spatial tracking and localization method based on improved SIFT feature points described in this invention, the feature point extraction includes the following steps: based on the SURF algorithm, the filtering process is accelerated using an integral image and a Hessian matrix is ​​constructed, the specific formula of which is as follows:

[0010]

[0011] p = (x, y)

[0012] Where H(p,σ) are the eigenvalues ​​of the Hessian matrix; x is the x-coordinate of a pixel in the image; y is the y-coordinate of a pixel in the image; σ is the standard deviation of the Gaussian filter; and p is the coordinate of a pixel in the image. Feature points are detected using the Hessian matrix, and the determinant and trace of the Hessian matrix are calculated using the following formulas:

[0013] det(H)=L xx (p,σ)L yy (p,σ)-(L xy (p,σ)) 2

[0014] trace(H) = L xx (p,σ)+L yy (p,σ)

[0015] Where det(H) is the determinant of the Hessian matrix; trace(H) is the trace of the Hessian matrix; if the difference between the square of the determinant of the Hessian matrix and the square of the trace of the Hessian matrix is ​​det(H) - trace(H), then... 2 If the value is greater than the first threshold, the corresponding pixel is determined to be a corner point and feature point extraction is performed; otherwise, the corresponding pixel is determined not to be a corner point and feature point extraction is not performed.

[0016] As a preferred embodiment of the binocular visual spatial tracking and localization method based on improved SIFT feature points described in this invention, the feature point matching includes the following steps: constructing a scale space using the Difference of Gaussian (DoG) pyramid based on a multi-scale matching method; matching feature points using a similarity metric to find the best matching point pair at different scales; performing a scale consistency check on the matched feature points; if the scale difference between two feature points is greater than a second threshold, the corresponding matching point pair is determined to be a mismatch, and feature point matching is performed again; otherwise, the corresponding matching point pair is determined to be the best matching point pair.

[0017] As a preferred embodiment of the binocular visual spatial tracking and localization method based on improved SIFT feature points described in this invention, the disparity value of the feature points is calculated using an adaptive window size disparity calculation method. The adaptive window size disparity calculation method includes the following steps: calculating the gradient of a local image region based on the local texture characteristics of the image; introducing parameters reflecting local image characteristics and calculating the size of the adaptive window based on the gradient of the local image region; matching pixels based on the size of the adaptive window and calculating the absolute difference and SAD of pixels within the window; optimizing the disparity value calculation using a graph cut algorithm and constructing the energy function of the graph cut algorithm; and calculating the disparity value of the feature points based on the energy function of the graph cut algorithm.

[0018] As a preferred embodiment of the binocular visual spatial tracking and localization method based on improved SIFT feature points described in this invention, the disparity value of the feature points is calculated using the following formula:

[0019]

[0020] Where, d p D represents the optimal disparity value for feature point p. p Let W(x,y) be the set of disparity values ​​for feature point p; W(x,y) be the size of the adaptive window; I lThe pixel values ​​of the left image; I r is the pixel value of the right image; p1 is the pixel adjacent to feature point p; N is the neighborhood system of all feature point p; λ is the smoothing parameter; D p1 Let be the set of disparity values ​​for pixel p1; τ is the threshold that controls the influence of disparity differences.

[0021] As a preferred embodiment of the binocular visual spatial tracking and localization method based on improved SIFT feature points described in this invention, the calculation of the feature point position involves triangulation using the disparity value and the baseline length of the camera to obtain the three-dimensional coordinates of the feature point; the triangulation calculation involves using the least squares method to calculate the triangulation equation, minimizing the reprojection error, and obtaining the three-dimensional coordinates of the feature point; the formula for calculating the three-dimensional coordinates of the feature point is as follows:

[0022]

[0023] Where X0 represents the initially calculated feature point location; B is the baseline length of the camera; x l x represents the coordinates of the feature point in the left image; r d represents the coordinates of the feature point in the right image; p X represents the optimal disparity value for feature point p; X represents the three-dimensional coordinates of the feature point. To minimize the parameters of the objective function; x i P represents a feature point in the i-th image. i P is the projection matrix of the i-th camera; i 3 This is the third row of the projection matrix for the i-th camera.

[0024] Secondly, to further address the security issues in visual tracking and positioning, this invention provides a binocular visual spatial tracking and positioning system based on improved SIFT feature points. The system includes: an image feature module for acquiring image data and camera parameters, and extracting and matching feature points based on the image data to obtain the optimal matching point pair; a feature coordinate module for calculating the disparity value of the feature points based on the optimal matching point pair, and calculating the position of the feature points based on the disparity value and camera parameters to obtain the three-dimensional coordinates of the feature points; and a tracking and positioning module for tracking the motion of the feature points based on their three-dimensional coordinates, tracking the camera's motion based on the obtained feature point motion information, obtaining and optimizing the camera's position and orientation, thereby achieving the tracking and positioning of the target object.

[0025] Thirdly, embodiments of the present invention provide a computer device, including a memory and a processor, wherein the memory stores a computer program, and the computer program, when executed by the processor, implements any step of the binocular visual spatial tracking and localization method based on improved SIFT feature points as described in the first aspect of the present invention.

[0026] Fourthly, embodiments of the present invention provide a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, it implements any step of the binocular visual spatial tracking and localization method based on improved SIFT feature points as described in the first aspect of the present invention.

[0027] The beneficial effects of this invention are as follows: By utilizing the SURF algorithm to extract key feature points and then employing a multi-scale matching method for feature point multi-scale matching, this invention achieves accurate extraction and high-quality matching of stable and salient features in images, laying the foundation for subsequent disparity calculation and 3D reconstruction. By introducing an adaptive window disparity calculation method and a graph cut algorithm energy function optimization method, this invention adapts to changes in local texture and depth of the image, effectively improving the accuracy and robustness of disparity values. By using disparity values ​​and camera baselines for triangulation calculation and employing the least squares method to optimize reprojection errors, this invention achieves high-precision reconstruction of the 3D coordinates of feature points, providing accurate 3D spatial information for camera motion tracking. Attached Figure Description

[0028] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Wherein:

[0029] Figure 1 This is a flowchart of the visual spatial tracking and localization of the target object in Example 1. Detailed Implementation

[0030] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0031] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0032] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0033] Example 1

[0034] Reference Figure 1 This is the first embodiment of the present invention, which provides a binocular vision spatial tracking and localization method based on improved SIFT feature points, including the following steps:

[0035] S1: Acquire image data and camera parameters and perform preprocessing, including feature point extraction and feature point matching of the image data.

[0036] Specifically, such as Figure 1 The diagram illustrates the tracking and positioning process of this invention. It involves acquiring image data and camera parameters, extracting feature points from the image data using the SURF algorithm, detecting feature points using a constructed Hessian matrix, and determining whether corresponding pixels are corner points. If a corner point is identified, feature point extraction is performed. A multi-scale matching method is then used to match the extracted feature points, obtaining the optimal matching point pair. A disparity calculation method with adaptive window size is used to calculate the disparity value of the optimal matching point pair. Triangulation is performed based on the disparity value and camera parameters to obtain the three-dimensional coordinates of the feature points, and motion tracking is performed to obtain the camera's position and orientation. Finally, multi-sensor fusion and adaptive filtering are used to optimize the camera's position and orientation, achieving target object tracking and positioning.

[0037] Preferably, the image data consists of images from the left and right cameras acquired using a binocular vision system.

[0038] Preferably, the camera parameter is the baseline length of the camera.

[0039] Specifically, the preprocessing includes the following steps: using a Gaussian filter to denoise the image data.

[0040] Enhance the contrast of image data based on histogram equalization or adaptive enhancement algorithms.

[0041] Camera calibration is used for color correction to improve the accuracy of feature point extraction and matching.

[0042] Preferably, feature point extraction includes the following steps: Based on the SURF algorithm, the filtering process is accelerated using integral images, and a Hessian matrix is ​​constructed. The specific formula for the Hessian matrix is ​​as follows:

[0043]

[0044] p = (x, y)

[0045] Where H(p,σ) are the eigenvalues ​​of the Hessian matrix; x is the x-coordinate of a pixel in the image; y is the y-coordinate of a pixel in the image; σ is the standard deviation of the Gaussian filter; and p is the coordinate of a pixel in the image.

[0046] Feature points are detected using the Hessian matrix. The determinant and trace of the Hessian matrix are calculated using the following formulas:

[0047] det(H)=L xx (p,σ)L yy (p,σ)-(L xy (p,σ)) 2

[0048] trace(H) = L xx (p,σ)+L yy (p,σ)

[0049] Where det(H) is the determinant of the Hessian matrix; and trace(H) is the trace of the Hessian matrix.

[0050] Specifically, if the difference between the determinant of the Hessian matrix and the square of the trace of the Hessian matrix is ​​det(H) - trace(H) 2 If the value is greater than the first threshold, the corresponding pixel is determined to be a corner point and feature point extraction is performed; otherwise, the corresponding pixel is determined not to be a corner point and feature point extraction is not performed.

[0051] Specifically, for the first threshold for judging corner points, feature points are extracted using different thresholds based on the application scenario. The threshold is then adjusted based on the extracted feature points to achieve the best results in the application scenario, thus obtaining the optimal first threshold.

[0052] Specifically, by calculating the determinant and trace of the Hessian matrix and comparing the difference with a first threshold, it accurately determines whether a pixel is a corner point, effectively identifies significant features in the image, eliminates unstable or inconspicuous feature points, and improves the quality and stability of feature points.

[0053] Preferably, feature point matching includes the following steps: Based on a multi-scale matching method, a scale space is constructed using the Difference of Gaussian (DoG) pyramid, and then feature points are extracted at different scales. The specific formula for the Difference of Gaussian (DoG) pyramid is as follows:

[0054] D(x,y,σ)=(G(x,y,kσ)-G(x,y,σ))*I(x,y)

[0055]

[0056] Where D(x,y,σ) is the difference-of-Gaussian image; G(x,y,σ) is the Gaussian blur function; I(x,y) is the input image; k is the scale factor; x is the x-coordinate of a pixel in the image; y is the y-coordinate of a pixel in the image; and σ is the scale.

[0057] Feature points are matched using a similarity metric to find the best matching point pairs at different scales. The specific formula for the similarity metric is as follows:

[0058]

[0059] Where d(p,q) is the Euclidean distance between two feature points; if the Euclidean distance d(p,q) between two feature points is minimized, then feature points p and q are determined to be the best matching pair; p is the coordinate vector of one feature point; q is the coordinate vector of the other feature point; n is the dimension of the coordinate vector; p i Let q be the i-th component of feature point p; i Let be the i-th component of feature point q.

[0060] A scale consistency check is performed on the matched feature points. If the scale difference between two feature points is greater than a second threshold, the corresponding matching point pair is determined to be a mismatch, and feature point matching is performed again. Otherwise, the corresponding matching point pair is determined to be the best matching point pair. The specific formula for the scale consistency check is as follows:

[0061]

[0062] Where Consistency(p,q) represents the scale difference between the two feature points; σ p σ is the scale of feature point p; q Let q be the scale of the feature point.

[0063] Specifically, by collecting a large amount of image data and performing feature point matching, the scale difference data of correct and incorrect matches are statistically analyzed and a probability distribution is formed. Then, the probability distribution is calculated to obtain the optimal second threshold, thereby improving the effectiveness of scale consistency checks.

[0064] Specifically, by introducing the Difference of Gaussian (DoG) pyramid and scale consistency check, feature points are extracted at different scales, enabling the algorithm to adapt to changes in the size of the target object in the image. By comparing the scale difference between two feature points with a second threshold, inconsistent matching point pairs are effectively filtered out, reducing the possibility of false matching and improving the accuracy of matching.

[0065] S2: Calculate the disparity value of the feature points based on the matching results of the feature points.

[0066] Preferably, a disparity calculation method that adapts to the window size is used to calculate the disparity value of feature points, adapting to different texture and depth variations.

[0067] Specifically, the disparity calculation method for adaptive window size includes the following steps: calculating the gradient of the local image region based on the local texture characteristics of the image, as shown in the following formula:

[0068]

[0069] Where G(x,y) is the gradient of the image at point (x,y); I(x,y) is the input image.

[0070] By introducing parameters that reflect local image characteristics, and calculating the size of the adaptive window based on the gradient of the local image region, the specific formula is as follows:

[0071]

[0072] Where W(x,y) is the size of the adaptive window; m is the baseline value of the window size; and α and β are parameters for adjusting the influence of local gradients.

[0073] Pixel matching is performed based on an adaptive window size, and the absolute difference and SAD of pixels within the window are calculated. The specific formula is as follows:

[0074]

[0075] Where SAD(x,y,D) is the sum of the absolute differences between pixels within the window; D is the set of disparity values, ranging from [D...]. min D max The settings should be configured according to the application scenario and specific circumstances.

[0076] The disparity calculation is optimized using the graph cut algorithm. An energy function for the graph cut algorithm is constructed to improve the accuracy of the disparity value. The specific formula is as follows:

[0077]

[0078] Where E(D) is the energy function of the graph cut algorithm; p1 is the pixel adjacent to feature point p; N is the neighborhood system of all feature point p; λ is the smoothing parameter; and τ is the threshold for controlling the influence of disparity difference.

[0079] The disparity value of the feature point is obtained by calculating the energy function based on the graph cut algorithm.

[0080] Specifically, the formula for calculating the disparity value of a feature point is as follows:

[0081]

[0082] Where, d p D represents the optimal disparity value for feature point p. p Let W(x,y) be the set of disparity values ​​for feature point p; W(x,y) be the size of the adaptive window; I l The pixel values ​​of the left image; I r is the pixel value of the right image; p1 is the pixel adjacent to feature point p; N is the neighborhood system of all feature point p; λ is the smoothing parameter; D p1 Let be the set of disparity values ​​for pixel p1; τ is the threshold that controls the influence of disparity differences.

[0083] Specifically, by introducing an adaptive window size disparity calculation method, the window size is automatically adjusted according to the gradient of the local region of the image. This allows for the use of small windows in texture-rich areas to preserve details, and large windows in areas with simple textures to enhance the reliability of matching, thereby improving the accuracy of disparity calculation. Furthermore, by introducing a graph cut algorithm to optimize the disparity value, considering the smoothness constraints and disparity differences between adjacent pixels, local erroneous disparity values ​​are corrected, thus improving the robustness of disparity calculation.

[0084] S3: Calculate the position of the feature point using the parallax value and camera parameters to obtain the three-dimensional coordinates of the feature point.

[0085] Preferably, the position of the feature point is calculated by triangulation using the disparity value and the baseline length of the camera to obtain the three-dimensional coordinates of the feature point.

[0086] Furthermore, the triangulation calculation uses the least squares method to calculate the triangulation equation, minimizing the reprojection error. That is, minimizing the error between the observed image points and the points on the image plane that are reprojected from the triangulated 3D points, thereby obtaining the 3D coordinates of the feature points and improving the accuracy of the 3D coordinates.

[0087] Specifically, the formula for calculating the three-dimensional coordinates of a feature point is as follows:

[0088]

[0089] Where X0 represents the initially calculated feature point location; B is the baseline length of the camera; x l x represents the coordinates of the feature point in the left image; r d represents the coordinates of the feature point in the right image; p X represents the optimal disparity value for feature point p; X represents the three-dimensional coordinates of the feature point. To minimize the parameters of the objective function; x i P represents a feature point in the i-th image. i P is the projection matrix of the i-th camera; i 3This is the third row of the projection matrix for the i-th camera.

[0090] Specifically, by using the least squares method to optimize the triangulation equation, the reprojection error is minimized, effectively reducing the 3D reconstruction error caused by image noise and matching error, and improving the accuracy of the 3D coordinates of feature points.

[0091] S4: Motion tracking is performed based on three-dimensional coordinates to obtain the camera's state.

[0092] Preferably, the camera's state includes its position and orientation.

[0093] Specifically, motion tracking includes the following steps: using the KLT algorithm (Kanade-Lucas-Tomasi) to analyze the position of feature points in the previous frame and the image of the current frame in consecutive image frames, estimating the optical flow of pixels around the feature points, performing motion tracking of the feature points, and obtaining the motion information of the feature points.

[0094] The camera's translation and rotation information are calculated based on the motion information of feature points to obtain the camera's position and orientation.

[0095] S5: Optimizes the camera's state to achieve target object tracking and positioning.

[0096] Preferably, optimizations include multi-sensor fusion and adaptive filtering.

[0097] Specifically, multi-sensor fusion combines binocular vision with inertial measurement units and GPS to provide more accurate and robust positioning results. By fusing information from multiple sensors, it overcomes the limitations of a single sensor and improves the accuracy and reliability of positioning.

[0098] Specifically, adaptive filtering adjusts the filtering parameters and weights according to changes in the environment and motion state, adapting to different scenarios and motion conditions, improving the algorithm's robustness to noise, motion blur, and occlusion interference, and providing more stable and reliable positioning results.

[0099] Specifically, by fusing inertial and GPS sensors, the limitations of a single visual sensor are overcome, improving the accuracy and reliability of positioning. Adjusting parameters according to environmental changes enhances the algorithm's robustness to noise, motion blur, and occlusion interference, resulting in more stable and reliable positioning results.

[0100] This embodiment also provides a binocular vision spatial tracking and positioning system based on improved SIFT feature points, including: an image feature module, used to acquire image data and camera parameters, and extract and match feature points based on the image data to obtain the best matching point pair; a feature coordinate module, used to calculate the disparity value of the feature points based on the best matching point pair, and calculate the position of the feature points based on the disparity value and camera parameters to obtain the three-dimensional coordinates of the feature points; and a tracking and positioning module, used to perform motion tracking of the feature points based on the three-dimensional coordinates of the feature points, and perform motion tracking of the camera based on the obtained motion information of the feature points to obtain the position and attitude of the camera and optimize it, thereby realizing the tracking and positioning of the target object.

[0101] This embodiment also provides a computer device applicable to the binocular visual spatial tracking and localization method based on improved SIFT feature points, including: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the binocular visual spatial tracking and localization method based on improved SIFT feature points as proposed in the above embodiment.

[0102] The computer device can be a terminal, comprising a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, carrier networks, NFC (Near Field Communication), or other technologies. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device's casing, or an external keyboard, touchpad, or mouse.

[0103] This embodiment also provides a storage medium storing a computer program that, when executed by a processor, implements the binocular visual spatial tracking and localization method based on improved SIFT feature points as proposed in the above embodiments. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Red-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0104] In summary, this invention utilizes the SURF algorithm to extract key feature points and then employs a multi-scale matching method to perform multi-scale matching of feature points, achieving accurate extraction and high-quality matching of stable and salient features in images, laying the foundation for subsequent disparity calculation and 3D reconstruction. By introducing an adaptive window disparity calculation method and a graph cut algorithm energy function optimization method, it adapts to local texture and depth changes in images, effectively improving the accuracy and robustness of disparity values. By using disparity values ​​and camera baselines for triangulation calculation and employing the least squares method to optimize reprojection errors, it achieves high-precision reconstruction of the 3D coordinates of feature points, providing accurate 3D spatial information for camera motion tracking.

[0105] Example 2

[0106] Referring to Tables 1 and 2, the second embodiment of the present invention is provided. This embodiment differs from the first embodiment in that, in order to verify its beneficial effects, experimental comparison data between the present invention and the prior art are provided.

[0107] This example utilizes a binocular vision system, including two synchronized high-resolution digital cameras with a resolution of 1920×1080 and a baseline length of 20 cm, to acquire image data. The method of this invention is then used to extract and match feature points. Based on the matching results, the disparity value of the feature points is calculated, and the disparity value is optimized using an adaptive window size disparity calculation method to obtain the three-dimensional coordinates of the feature points. This enables the tracking and positioning of the target object. Table 1 shows the tracking and positioning data for different objects.

[0108] Table 1 Tracking and positioning data for different objects

[0109]

[0110] For the data of the different objects mentioned above, the objects were tracked and located using the method of this invention, the monocular vision localization algorithm based on feature points, and the binocular vision localization algorithm based on the direct method, respectively. The specific localization data is shown in Table 2.

[0111] Table 2 Comparison of Tracking and Positioning Error Data for Different Objects

[0112]

[0113] As can be seen from the table above, when the distance to the object, the rate of change of the target scale, the change of the viewing angle, the motion blur, and the occlusion ratio all gradually increase, the present invention effectively reduces the error of the target localization data, indicating that the present invention can adapt to a wider range of scale and viewing angle changes and has stronger tracking capabilities. Furthermore, through the comparison between the present invention and the prior art, it can be seen that the present invention, by introducing multi-scale feature point extraction and scale consistency checks, can adapt to changes in the scale of the target object, reduce mismatches caused by scale differences, and by using disparity calculation with adaptive window size and graph cut algorithm optimization, can adaptively adjust the window size according to image features, improve the accuracy of disparity calculation, and thus achieve more accurate target object tracking and localization.

[0114] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. Binocular visual space tracking and positioning method based on improved SIFT feature points, Features: include: Collecting image data and camera parameters and performing preprocessing, and performing feature point extraction and feature point matching on the image data; Calculate the disparity value of the feature point according to the matching result of the feature point; Calculating the position of the feature point using the disparity value and the camera parameter to obtain the three-dimensional coordinates of the feature point; Perform motion tracking based on the three-dimensional coordinates to obtain a state of the camera; The state of the camera is optimized to achieve tracking and positioning of the target object.

2. The binocular visual space tracking and positioning method based on improved SIFT feature points as claimed in claim 1, Features: The image data includes a left camera image and a right camera image; The camera parameter is the baseline length of the camera.

3. The binocular visual space tracking and positioning method based on improved SIFT feature points as claimed in claim 2, Features: The feature point extraction comprises the following steps: Based on the SURF algorithm, the integral image is used to accelerate the filtering process and construct the Hessian matrix. The specific formula of the Hessian matrix is ​​as follows: p=(x,y) Where H(p,σ) is the eigenvalue of the Hessian matrix; x is the horizontal coordinate of the pixel in the image; y is the vertical coordinate of the pixel in the image; σ is the standard deviation of the Gaussian filter; p is the coordinate of the pixel in the image; The Hessian matrix is ​​used to detect feature points and calculate the determinant and trace of the Hessian matrix. The specific formula is as follows: det(H)=L xx (p,σ)L yy (p,σ)-(L xy (p,σ)) 2 trace(H)=L xx (p,σ)+L yy (p,s) Where det(H) is the determinant of the Hessian matrix; trace(H) is the trace of the Hessian matrix; If the difference between the determinant of the Hessian matrix and the square of the trace of the Hessian matrix is ​​det(H)-trace(H) 2 If the value is greater than the first threshold, the corresponding pixel is determined to be a corner point and feature point extraction is performed; otherwise, the corresponding pixel is determined not to be a corner point and feature point extraction is not performed.

4. The binocular visual space tracking and positioning method based on improved SIFT feature points as claimed in claim 3, Features: The feature point matching comprises the following steps: Based on the multi-scale matching method, the Gaussian difference DoG pyramid is used to construct the scale space; Use similarity measurement to match feature points and find the best matching point pairs at different scales; The scale consistency of the matched feature points is checked. If the scale difference between the two feature points is greater than the second threshold, the corresponding matching point pair is determined to be a mismatch and the feature point matching is performed again. Otherwise, the corresponding matching point pair is determined to be the best matching point pair.

5. The binocular visual space tracking and positioning method based on improved SIFT feature points as claimed in claim 4, Features: Calculating the disparity value of the feature point using a disparity calculation method with an adaptive window size; The disparity calculation method of the adaptive window size comprises the following steps: Calculate the gradient of the local image area according to the local texture characteristics of the image; Introduce parameters that reflect local image characteristics, calculate based on the gradient of the local image area, and obtain the size of the adaptive window; Matching pixels based on the size of the adaptive window, and calculating the absolute difference and SAD of the pixels in the window; The graph cut algorithm is used to optimize the calculation of disparity values ​​and the energy function of the graph cut algorithm is constructed; The energy function of the graph cut algorithm is used for calculation to obtain the disparity value of the feature point.

6. The binocular visual space tracking and positioning method based on improved SIFT feature points as claimed in claim 5, Features: The calculation formula of the disparity value of the feature point is as follows: Among them, d p is the optimal disparity value of feature point p; D p is the disparity value set of feature point p; W(x,y) is the size of the adaptive window; I l is the pixel value of the left image; I r is the pixel value of the right image; p 1 is the pixel point adjacent to the feature point p; N is the neighborhood system of all feature points p; λ is the smoothing parameter; D p1 is the pixel p 1 A set of disparity values; τ is the threshold for controlling the influence of disparity difference.

7. The binocular visual space tracking and positioning method based on improved SIFT feature points as claimed in claim 6, Features: The position of the feature point is calculated by triangulating the position of the feature point using the disparity value and the baseline length of the camera to obtain the three-dimensional coordinates of the feature point. The triangulation calculation is to calculate the triangulation equation using the least square method, minimize the reprojection error, and obtain the three-dimensional coordinates of the feature points; The calculation formula of the three-dimensional coordinates of the feature points is as follows: Among them, X 0 is the position of the feature point calculated initially; B is the baseline length of the camera; x l is the coordinate of the feature point in the left image; x r is the coordinate of the feature point in the right image; d p is the optimal disparity value of feature point p; X is the three-dimensional coordinate of the feature point; is the parameter to minimize the objective function; x i is the feature point in the i-th image; P i is the projection matrix of the i-th camera; is the third row of the projection matrix of the i-th camera.

8. A binocular visual space tracking and positioning system based on improved SIFT feature points, based on the binocular visual space tracking and positioning method based on improved SIFT feature points according to any one of claims 1 to 7, Features: include, Image feature module, used to collect image data and camera parameters, and perform feature point extraction and feature point matching based on image data to obtain the best matching point pair; The feature coordinate module is used to calculate the disparity value of the feature point according to the best matching point pair, and calculate the position of the feature point according to the disparity value and the camera parameters to obtain the three-dimensional coordinates of the feature point; The tracking and positioning module is used to track the motion of feature points according to the three-dimensional coordinates of the feature points, track the motion of the camera based on the obtained motion information of the feature points, obtain the position and posture of the camera and optimize it to achieve tracking and positioning of the target object.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program. Features: When the processor executes the computer program, the steps of the binocular visual space tracking and positioning method based on improved SIFT feature points according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, Features: When the computer program is executed by a processor, the steps of the binocular visual space tracking and positioning method based on improved SIFT feature points according to any one of claims 1 to 7 are implemented.

Citation Information

Cited By

  • Animal trace monitoring method and system based on machine vision

    CN119810742A