A method and system for tracking and positioning a moving target in space using binocular vision modeling

Abstract

The application belongs to the technical field of optical stereoscopic imaging, and particularly relates to a binocular vision modeling space trajectory tracking positioning method and system for a moving target, which comprises binocular device construction, device internal and external parameter correction, scene optical stereoscopic image modeling, optical stereoscopic image and two-dimensional optical image acquisition device pixel matching, two-dimensional optical image moving target identification, moving target space positioning, and moving target trajectory tracking. The binocular vision acquires optical images to establish a three-dimensional optical stereoscopic image scene model, and the monocular acquires optical images to track a target. The target is positioned according to the moving target only, so as to reduce the calculation amount in the moving target trajectory tracking positioning process and improve the real-time performance of the moving target trajectory tracking positioning.

Description

Technical Field

[0001] This invention belongs to the field of optical stereo imaging technology, specifically relating to a method and system for tracking and locating the spatial trajectory of a moving target using binocular vision modeling. Background Technology

[0002] Binocular stereo vision fuses images acquired by both eyes and observes the differences between them, enabling us to obtain a clear sense of depth, establish correspondences between features, and match the image points of the same spatial physical point in different images. Binocular stereo vision measurement methods have advantages such as high efficiency, suitable accuracy, simple system structure, and low cost, making them ideal for online, non-contact product inspection and quality control in manufacturing environments. In measuring moving objects (including animal and human shapes), stereo vision methods are even more efficient because image acquisition is completed instantaneously. Binocular stereo vision systems are one of the key technologies in computer vision, and acquiring distance information from three-dimensional spatial scenes is a fundamental aspect of computer vision research.

[0003] Binocular vision technology, as a branch of machine vision, is gradually being used in near-range 3D imaging as the technology matures, and is increasingly being applied to daily life as an emerging 3D imaging technology. Binocular stereo vision is based on the principle of parallax and is a form of passive ranging, possessing non-contact capability. It requires no signal emission to the target; target distance detection can be completed by synchronously acquiring and processing target images in natural scenes, and measurements can be repeated multiple times. However, the 3D modeling process of binocular vision is relatively slow, and it cannot track rapidly moving targets in space in a timely manner, thus increasing the difficulty of real-time analysis of moving target localization. Summary of the Invention

[0004] The purpose of this invention is to provide a method for tracking and locating the spatial trajectory of a moving target using binocular vision modeling. This method uses binocular vision to acquire optical images to create a three-dimensional optical stereo image to establish a scene model, and combines this with monocular optical images to track the target. By tracking the target, the method calculates the position of the moving target only, thereby solving the problems mentioned in the background art.

[0005] To achieve the above objectives, the present invention adopts the following technical solution: a binocular vision modeling method for tracking and locating the spatial trajectory of a moving target, comprising: S1: placing two sets of optical image acquisition devices horizontally according to equipment setup guidelines, and setting the baseline distance between the two sets of optical image acquisition devices to complete the setup of the binocular vision device; S2: performing multiple optical image acquisitions on a calibration board using the two sets of optical image acquisition devices, and correcting internal and external parameters based on the acquired optical images; S3: using the two sets of optical image acquisition devices to acquire optical images of the scene, and performing scene optical stereoscopic image modeling using the acquired two sets of optical images; S4: selecting one of the two sets of optical images as a two-dimensional optical image acquisition device. S5: The two-dimensional optical image is matched pixel by pixel with the optical stereo image; S6: The two-dimensional optical image acquisition device is used to acquire optical images in real time and compare them with multiple acquired optical images to identify moving targets; S7: Moving target identification and tracking is performed based on the two-dimensional optical image acquired by the two-dimensional optical image acquisition device. The binocular vision device is used to acquire optical image data of the moving target at the same time. The corresponding pixels of the moving target are matched with stereo images through a matching algorithm to obtain the spatial position of the pixels; S8: The spatial positioning of the moving target acquired by the two-dimensional optical image is fused with the scene optical stereo image. Steps S5 and S6 are repeated to record the movement trajectory of the moving target over time and to track the trajectory of the moving target.

[0006] Preferably, the equipment setup criteria include: the two sets of optical image acquisition devices are placed horizontally relative to each other and their optical axes are uniformly parallel, and the two sets of optical image acquisition devices can acquire optical images simultaneously without delay. The optical images acquired by the two sets of optical image acquisition devices have more than half of their overlap, and the required acquisition scene range is within the overlap portion.

[0007] Preferably, the step of acquiring multiple optical images of the calibration board using two sets of optical image acquisition devices includes: ensuring that the image of the calibration board is completely within the optical images acquired by the two sets of optical image acquisition devices during the shooting process, and acquiring multiple sets of optical images to obtain internal and external parameters; wherein, the internal parameters include the optical device distortion coefficient and pixel focal length, and the external parameters include translation vector and rotation vector.

[0008] Preferably, the step of correcting internal and external parameters based on the acquired optical images includes: correcting image distortion of the optical images by means of the optical equipment distortion coefficient and pixel focal length according to the calibrated optical equipment distortion coefficient; and correcting the relative position of the two sets of optical image acquisition devices by means of translation and rotation in a three-dimensional Cartesian coordinate system by means of externally acquired parameters.

[0009] Preferably, the process of acquiring two sets of optical images to perform scene optical stereo image modeling includes: obtaining distortion-corrected images from the two optical image acquisition devices respectively; matching pixels using the SURF image matching algorithm; and inverting the three-dimensional point cloud using the SGBM algorithm to perform optical stereo image modeling of the scene.

[0010] Preferably, the step of matching the acquired two-dimensional optical image with the optical stereo image includes: selecting any one of the optical image acquisition devices in the binocular vision device as a monocular two-dimensional optical image acquisition device, and matching the coordinates of the pixels in the same coordinate system by using the external parameter information obtained between the binocular vision device and the monocular device and the horizontal and pitch angle information obtained from the internal parameters.

[0011] Preferably, the identification of moving targets includes: the monocular device continuously acquiring two-dimensional optical images; comparing multiple sets of acquired optical images pairwise; and performing image subtraction and image matching through grayscale conversion and binary image contour extraction to identify moving targets in the monocular acquired two-dimensional optical images.

[0012] Preferably, the moving target spatial positioning includes: after detecting and identifying the moving target, activating the binocular vision device to acquire optical images of the moving target; extracting the corresponding pixels of the identified moving target using the SURF image matching algorithm based on the optical images acquired by the binocular vision device; and spatially positioning the moving target using pixel matching based on the extracted corresponding pixels of the moving target and the internal and external parameters obtained by the calibration of the binocular vision device, thereby determining the three-dimensional coordinates of the moving target in three-dimensional space.

[0013] Preferably, the moving target trajectory tracking includes: continuously repeating steps S5 and S6 over time; acquiring the three-dimensional coordinates corresponding to the three-dimensional space where the moving target is located in real time according to a set time interval; fusing the moving target's corresponding coordinate position in the three-dimensional space with the scene optical stereo image acquired by the binocular vision device; fusing the three-dimensional model corresponding to the moving target into the scene optical stereo image through coordinate system matching; updating it in real time over time; and saving the moving target's trajectory data over time.

[0014] Technical effects and advantages of the present invention: The binocular vision modeling method for tracking and locating the spatial trajectory of a moving target proposed in this invention has the following advantages compared with the prior art:

[0015] By acquiring optical images through binocular vision to build a 3D optical stereo imagery scene model, and combining it with monocular acquired optical images for target tracking, the tracking target calculates its position only based on the moving target, thereby reducing the amount of computation in the moving target trajectory tracking and positioning process and improving the real-time performance of moving target trajectory tracking and positioning. Using a monocular device for moving target recognition has a faster response speed and occupies less memory. By recognizing the moving target pixels only through monocular vision for 3D spatial positioning and fusing it with the scene optical stereo imagery, the fused effect can ensure that the original pixels in the scene are not lost. Attached Figure Description

[0016] Figure 1 This is a flowchart of a method for tracking and locating the spatial trajectory of a moving target using binocular vision modeling, as described in an embodiment of the present invention. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The specific embodiments described herein are merely used to explain the present invention and are not intended to limit the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] This invention provides a method for tracking and locating the spatial trajectory of a moving target using binocular vision modeling. The method uses binocular vision to acquire optical images to create a three-dimensional optical stereo image to build a scene model, and combines this with monocular optical images to track the target. By tracking the target, the method calculates the position of the moving target only based on the moving target, thereby reducing the amount of computation in the trajectory tracking and locating process and improving the real-time performance of the trajectory tracking and locating of the moving target.

[0019] like Figure 1 The method for tracking and locating the spatial trajectory of a moving target using binocular vision modeling, as shown, includes the following steps:

[0020] S1: Place the two sets of optical image acquisition devices horizontally according to the equipment setup guidelines, and set the baseline distance between the two sets of optical image acquisition devices to complete the setup of the binocular vision device.

[0021] Specifically, the equipment setup guidelines include: two sets of optical image acquisition devices should be placed horizontally relative to each other and with their optical axes parallel, and the two sets of optical image acquisition devices should be able to acquire optical images simultaneously without delay.

[0022] S2: Multiple optical image acquisitions are performed on the calibration board using two sets of optical image acquisition devices, and internal and external parameters are corrected based on the acquired optical images.

[0023] Specifically, the optical images acquired by the two sets of optical image acquisition devices have more than half of their overlap, and the required acquisition scene range is within the overlap area.

[0024] Furthermore, the calibration board is subjected to multiple optical image acquisitions using two sets of optical image acquisition devices, including: during the calibration board shooting process, its image is completely located in the optical images acquired by the two sets of optical image acquisition devices, and multiple sets of optical images are captured to obtain internal and external parameters.

[0025] The internal parameters include the optical device distortion coefficient and pixel focal length, while the external parameters include the translation vector and rotation vector.

[0026] In addition, internal and external parameters are corrected based on the acquired optical images, including: correcting image distortion by using the optical equipment distortion coefficient and pixel focal length based on the calibrated optical equipment distortion coefficient; and correcting the relative position of the two sets of optical image acquisition devices by translating and rotating them in a three-dimensional Cartesian coordinate system using externally acquired parameters.

[0027] S3: Use two sets of optical image acquisition devices to acquire optical images of the scene, and use the acquired two sets of optical images to create a stereoscopic optical image model of the scene.

[0028] Specifically, based on the distortion-corrected images acquired by the two optical image acquisition devices, the pixels are matched using the SURF image matching algorithm; the 3D point cloud is inverted using the SGBM algorithm, and then the scene is modeled using optical stereo images.

[0029] S4: Select one of the two sets of optical images as a two-dimensional optical image acquisition device, and perform pixel matching between the acquired two-dimensional optical image and the optical stereo image.

[0030] Specifically, any optical image acquisition device in the binocular vision system is selected as a monocular two-dimensional optical image acquisition device. The external parameter information obtained between the binocular vision system and the monocular device, as well as the horizontal and pitch angle information obtained from the internal parameters, are used to achieve one-to-one coordinate matching of pixels in the same coordinate system.

[0031] S5: Use a two-dimensional optical image acquisition device to acquire optical images in real time and compare them with multiple acquired optical images to identify moving targets.

[0032] Specifically, the monocular device continuously acquires two-dimensional optical images. By comparing multiple sets of acquired optical images in pairs, and through grayscale conversion and binary image contour extraction, image subtraction and image matching are performed to enable the identification of moving targets in the monocular acquired two-dimensional optical images.

[0033] S6: Based on the two-dimensional optical images acquired by the two-dimensional optical image acquisition device, the moving target is identified and tracked. The moving target is simultaneously acquired by a binocular vision device. The corresponding pixels of the moving target are matched with stereo images by a matching algorithm to obtain the spatial position of the pixels.

[0034] Specifically, moving target spatial localization includes: after detecting and identifying the moving target, using a binocular vision device to acquire optical images of the moving target; extracting the corresponding pixels of the identified moving target using the SURF image matching algorithm based on the optical images acquired by the binocular vision device; and using the extracted corresponding pixels of the moving target, along with the internal and external parameters obtained from the calibration of the binocular vision device, spatially locating the moving target through pixel matching to determine its three-dimensional coordinates in three-dimensional space.

[0035] S7: The spatial positioning of the moving target is acquired by acquiring two-dimensional optical images and fused with the scene optical stereo images. Steps S5 and S6 are repeated to record the movement trajectory of the moving target over time and to track the moving target trajectory.

[0036] Specifically, moving target trajectory tracking includes: continuously repeating steps S5 and S6 over time to acquire the three-dimensional coordinates of the moving target in the three-dimensional space in real time according to a set time interval; fusing the moving target's corresponding coordinate position in the three-dimensional space with the scene optical stereo image acquired by the binocular vision device; fusing the moving target's corresponding three-dimensional model into the scene optical stereo image through coordinate system matching; updating in real time over time; and saving the moving target's trajectory data over time.

[0037] This invention employs a monocular device for moving target recognition, resulting in faster response speed and lower memory usage. By recognizing moving target pixels using only a monocular camera for 3D spatial localization and fusing it with scene optical stereo imagery, the fused imagery retains all original pixels from the scene. Binocular vision acquires optical stereo imagery for moving target tracking and localization, while using a monocular device to acquire optical imagery for 3D spatial localization provides even faster moving target recognition and improves real-time performance during moving target tracking.

[0038] In other embodiments, the traditional camera calibration method is preferred, but other active vision camera calibration methods, camera self-calibration methods, and zero-loss camera calibration methods are also acceptable.

[0039] In other embodiments, the stereo matching algorithm for binocular vision optical stereo images is preferably the SGBM algorithm, but it can be replaced by the SAD matching algorithm, BM algorithm, GC algorithm, etc.

[0040] In other embodiments, the SURF algorithm is preferred for matching the pixel of a moving target when a monocular device acquires multiple two-dimensional optical images, but algorithms such as SIFT can be used instead.

[0041] On the other hand, the present invention also proposes a binocular vision modeling system for tracking and locating the spatial trajectory of a moving target, which includes: an acquisition device module, a parameter correction module, an image modeling module, a pixel matching module, a target recognition module, a spatial position acquisition module, and a target tracking module.

[0042] For example, the acquisition device module is used to place two sets of optical image acquisition devices horizontally according to the equipment construction guidelines, and set the baseline distance between the two sets of optical image acquisition devices to complete the construction of the binocular vision device.

[0043] For example, the parameter correction module is used to perform multiple optical image acquisitions on the calibration board using the two sets of optical image acquisition devices, and to perform internal and external parameter corrections based on the acquired optical images;

[0044] For example, the image modeling module is used to acquire optical images of the scene using two sets of optical image acquisition devices, and to perform optical stereoscopic image modeling of the scene using the acquired two sets of optical images.

[0045] For example, the pixel matching module is used to select one of the two sets of optical images as a two-dimensional optical image acquisition device, and perform pixel matching between the acquired two-dimensional optical image and the optical stereo image.

[0046] For example, the target recognition module is used to acquire optical images in real time using the two-dimensional optical image acquisition device, and compare them with multiple acquired optical images to identify moving targets;

[0047] For example, the spatial location acquisition module is used to identify and track moving targets based on the two-dimensional optical images acquired by the two-dimensional optical image acquisition device. The binocular vision device is used to simultaneously acquire optical image data of the moving target, and a matching algorithm is used to perform stereo image matching on the pixels corresponding to the moving target to obtain the spatial location corresponding to the pixels.

[0048] For example, the target tracking module is used to acquire the spatial positioning of the moving target through the two-dimensional optical image and fuse it with the scene optical stereo image, record the moving target's trajectory over time, and perform moving target trajectory tracking.

[0049] The aforementioned acquisition device module, parameter correction module, image modeling module, pixel matching module, target recognition module, spatial position acquisition module, and target tracking module are also used to perform other functions of the binocular vision modeling method for tracking and locating the spatial trajectory of moving targets as described above, which will not be elaborated here.

[0050] In this embodiment, a scene model is established by acquiring optical images with binocular vision to create three-dimensional optical stereo images, and target tracking is performed by combining them with optical images acquired with monocular vision. By tracking the target, the position of the target is calculated only based on the moving target, thereby reducing the amount of computation in the moving target trajectory tracking and positioning process and improving the real-time performance of moving target trajectory tracking and positioning.

[0051] Finally, it should be noted that the above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for tracking and locating the spatial trajectory of a moving target using binocular vision modeling, characterized in that, include: S1: Place the two sets of optical image acquisition devices horizontally according to the equipment setup guidelines, and set the baseline distance between the two sets of optical image acquisition devices to complete the setup of the binocular vision device. S2: The calibration board is subjected to multiple optical image acquisitions using the two sets of optical image acquisition devices, and the internal and external parameters are corrected based on the acquired optical images. S3: Use two sets of optical image acquisition devices to acquire optical images of the scene, and use the acquired two sets of optical images to create an optical stereoscopic image model of the scene. S4: Select one of the two sets of optical images as a two-dimensional optical image acquisition device, and perform pixel matching between the acquired two-dimensional optical image and the optical stereo image; S5: Use the two-dimensional optical image acquisition device to acquire optical images in real time, and compare them with multiple acquired optical images to identify moving targets; S6: Based on the two-dimensional optical images acquired by the two-dimensional optical image acquisition device, moving target recognition and tracking are performed. The binocular vision device is used to simultaneously acquire optical image data of the moving target. The corresponding pixels of the moving target are matched with stereo images through a matching algorithm to obtain the spatial position of the pixels. S7: The spatial positioning of the moving target is acquired by the two-dimensional optical image and fused with the scene optical stereo image. Steps S5 and S6 are repeated to record the movement trajectory of the moving target over time and to track the moving target trajectory. The step of matching pixels of the acquired two-dimensional optical images with optical stereo images includes: selecting any optical image acquisition device in the binocular vision device as a monocular acquisition two-dimensional optical image device, and matching the pixels one-to-one in the same coordinate system by using the external parameter information obtained between the binocular vision device and the monocular device and the horizontal and pitch angle information obtained from the internal parameters. The identification of moving targets includes: the monocular device continuously acquiring two-dimensional optical images; comparing multiple sets of acquired optical images pairwise; and performing image subtraction and image matching through grayscale conversion and binary image contour extraction to identify moving targets in the monocular acquired two-dimensional optical images. The moving target spatial localization includes: after detecting and identifying the moving target, activating the binocular vision device to acquire optical images of the moving target; extracting the corresponding pixels of the identified moving target and the optical images acquired by the binocular vision device using the SURF image matching algorithm; and spatially locating the moving target using pixel matching based on the extracted corresponding pixels of the moving target and the internal and external parameters obtained by the calibration of the binocular vision device, thereby determining the three-dimensional coordinates of the moving target in three-dimensional space.

2. The method for tracking and locating the spatial trajectory of a moving target using binocular vision modeling according to claim 1, characterized in that, The equipment setup guidelines include: the two sets of optical image acquisition devices are placed horizontally relative to each other and their optical axes are parallel, and the two sets of optical image acquisition devices can acquire optical images simultaneously without delay; The optical images acquired by the two sets of optical image acquisition devices have more than half of their overlap, and the required acquisition scene range is within the overlap.

3. The method for tracking and locating the spatial trajectory of a moving target using binocular vision modeling according to claim 1, characterized in that, The process of acquiring multiple optical images of the calibration board using two sets of optical image acquisition devices includes: During the calibration board shooting process, its image is completely located in the optical images acquired by the two sets of optical image acquisition devices, and multiple sets of optical images are captured to obtain internal and external parameters. The internal parameters include the optical device distortion coefficient and the pixel focal length, while the external parameters include the translation vector and the rotation vector.

4. The method for tracking and locating the spatial trajectory of a moving target using binocular vision modeling according to claim 3, characterized in that, The step of correcting internal and external parameters based on the acquired optical images includes: The optical image is corrected for distortion based on the optical device distortion coefficient and the pixel focal length. The relative positions of the two sets of optical image acquisition devices are corrected by translating and rotating them in a three-dimensional Cartesian coordinate system using externally acquired parameters.

5. The method for tracking and locating the spatial trajectory of a moving target using binocular vision modeling according to claim 4, characterized in that, The process of acquiring two sets of optical images to perform scene optical stereoscopic image modeling includes: Based on the distortion-corrected images acquired by the two optical image acquisition devices respectively; Pixels are matched using the SURF image matching algorithm; The SGBM algorithm is used to invert 3D point clouds, and then optical stereoscopic image modeling of the scene is performed.

6. The method for tracking and locating the spatial trajectory of a moving target using binocular vision modeling according to claim 1, characterized in that, The moving target trajectory tracking includes: The steps S5 and S6 are repeated continuously over time, and the three-dimensional coordinates of the moving target in the three-dimensional space are acquired in real time according to the set time interval. The moving target's coordinate position in three-dimensional space is fused with the scene's optical stereo image acquired by the binocular vision device. The 3D model corresponding to the moving target is fused into the scene's optical stereo image through coordinate system matching. The data is updated in real time over time, and the moving target's trajectory is saved based on the time elapsed.

7. A binocular vision modeling system for tracking and locating the spatial trajectory of a moving target, characterized in that, include: The acquisition device module is used to place two sets of optical image acquisition devices horizontally according to the equipment construction guidelines, and to set the baseline distance between the two sets of optical image acquisition devices to complete the construction of the binocular vision device. The parameter correction module is used to perform multiple optical image acquisitions on the calibration board using the two sets of optical image acquisition devices, and to correct internal and external parameters based on the acquired optical images. The image modeling module is used to acquire optical images of the scene using two sets of optical image acquisition devices, and to perform optical stereoscopic image modeling of the scene using the acquired two sets of optical images. The pixel matching module is used to select one of the two sets of optical images as a two-dimensional optical image acquisition device, and perform pixel matching between the acquired two-dimensional optical image and the optical stereo image. The target recognition module is used to acquire optical images in real time using the two-dimensional optical image acquisition device, and compare them with multiple acquired optical images to identify moving targets. The spatial location acquisition module is used to identify and track moving targets based on the two-dimensional optical images acquired by the two-dimensional optical image acquisition device. The binocular vision device is used to simultaneously acquire optical image data of the moving target, and a matching algorithm is used to perform stereo image matching on the pixels corresponding to the moving target to obtain the spatial location of the pixels. The target tracking module is used to acquire the spatial positioning of the moving target through the two-dimensional optical image and fuse it with the scene optical stereo image, record the movement trajectory of the moving target over time, and perform moving target trajectory tracking. The step of matching pixels of the acquired two-dimensional optical images with optical stereo images includes: selecting any optical image acquisition device in the binocular vision device as a monocular acquisition two-dimensional optical image device, and matching the pixels one-to-one in the same coordinate system by using the external parameter information obtained between the binocular vision device and the monocular device and the horizontal and pitch angle information obtained from the internal parameters. The identification of moving targets includes: the monocular device continuously acquiring two-dimensional optical images; comparing multiple sets of acquired optical images pairwise; and performing image subtraction and image matching through grayscale conversion and binary image contour extraction to identify moving targets in the monocular acquired two-dimensional optical images. The moving target spatial localization includes: after detecting and identifying the moving target, activating the binocular vision device to acquire optical images of the moving target; extracting the corresponding pixels of the identified moving target and the optical images acquired by the binocular vision device using the SURF image matching algorithm; and spatially locating the moving target using pixel matching based on the extracted corresponding pixels of the moving target and the internal and external parameters obtained by the calibration of the binocular vision device, thereby determining the three-dimensional coordinates of the moving target in three-dimensional space.

Images