System and method for triangulation positioning based on multi-camera vision

Abstract

The invention discloses a triangulation positioning system and method based on multi-camera vision. The system comprises cameras, a calibration object and a tracking object. Firstly, internal parameters of a camera are calibrated, then external parameters of the camera are obtained and calculated according to a calibration object, and finally, triangulation positioning is carried out based on the internal parameters and the external parameters of the camera. Through accurate internal and external parameter calibration and multi-camera visual angle fusion calculation, high-precision three-dimensional positioning can be realized under reasonable conditions, and the accuracy and robustness of three-dimensional positioning are effectively improved. According to the method, when the imaging resolution of the camera is high and the calibration precision is sufficient, the positioning error can reach the millimeter level, and the high-precision positioning requirements of industrial detection, precise assembly, robot operation and the like can be met. Based on a multi-camera vision method, a plurality of cameras at different positions and at different angles are used for shooting a target, calibration parameters and a triangulation principle are combined, and high positioning precision can be obtained on the premise that expensive hardware is not added.

Description

Technical Field

[0001] This invention belongs to the field of video image processing, specifically relating to a triangulation and positioning system and method based on multi-camera vision. Background Technology

[0002] With the development of fields such as automated manufacturing, robot navigation, precision assembly, and augmented reality, the demand for high-precision 3D positioning of target objects in space is increasing. Traditional positioning methods include solutions based on LiDAR, structured light, and monocular or binocular vision.

[0003] While solutions such as lidar and structured light offer high precision, they are expensive, bulky, and complex to maintain, making them unsuitable for certain space-constrained or cost-sensitive scenarios.

[0004] Monocular vision lacks depth information and cannot achieve high-precision 3D positioning independently; when the baseline length is limited, the depth error of binocular vision increases significantly with distance, making it difficult to meet the millimeter-level accuracy requirements. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to provide a triangulation positioning system and method based on multi-camera vision, which solves the problems of insufficient accuracy and stability of existing visual positioning methods.

[0006] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:

[0007] A triangulation and positioning system based on multi-camera vision includes cameras, calibration objects, and tracking objects. The cameras consist of three cameras with large baseline differences but whose lenses face the same area. Multiple calibration objects are stationary at different locations within the overlapping fields of view of the three cameras, used for extrinsic parameter calibration. A single spherical tracking object is placed at any location within the overlapping field of view of the cameras for tracking and positioning. First, the camera intrinsic parameters are calibrated. Then, the camera extrinsic parameter data is obtained and calculated based on the calibration objects. Finally, triangulation and positioning are performed based on the camera intrinsic and extrinsic parameters.

[0008] The calibration objects consist of four yellow spherical calibration objects, each with a height of 0 in the custom world coordinate system.

[0009] The tracker is a yellow, spherical tracker.

[0010] The triangulation positioning method based on the system includes the following steps:

[0011] Step 1: Camera intrinsic parameter calibration, obtain and save the intrinsic parameter matrix and distortion coefficients for each camera;

[0012] Step 2: Obtain camera extrinsic data. In each camera's video frame, identify and track all yellow spherical markers based on a color threshold. Calculate the optimal point of each yellow spherical marker in the image based on multi-frame data. Then, manually input the coordinates of each yellow spherical marker in a custom world coordinate system and match and record the pixel coordinates with the real coordinates in the custom world coordinate system.

[0013] Step 3: Camera extrinsic parameter calculation. Using the pixel coordinates and real-world coordinates obtained in Step 2, calculate and save the extrinsic parameter data for each camera.

[0014] Step 4: Triangulation and positioning. In each video frame of each camera, the same yellow spherical marker is identified and tracked. Based on the multi-frame data, the optimal tracking point of the yellow spherical marker on each camera image is calculated. Based on the intrinsic and extrinsic parameters of each camera, triangulation calculation is performed on the optimal tracking point, and finally the three-dimensional coordinates of the point in the custom world coordinate system are output.

[0015] In step 1, the intrinsic parameters of each camera are calibrated using a checkerboard pattern, which specifically includes the following steps:

[0016] Step 1.1: Place the checkerboard pattern within the camera's field of view and capture images of the checkerboard pattern from different positions and angles;

[0017] Step 1.2: Convert the acquired image to grayscale;

[0018] Step 1.3: Detect corner points in each frame of the image and perform sub-pixel level precision;

[0019] Step 1.4: Obtain the camera's intrinsic parameter matrix and distortion coefficients, and save them.

[0020] Step 2 specifically includes the following steps:

[0021] Step 2.1: Convert the current frame image to HSV format;

[0022] Step 2.2: Find the regions H(5,30), S(77,255), and V(160,255) in the image that match the HSV range, and find the target color contour;

[0023] Step 2.3: Traverse all target color contours. For those that meet the area requirements, calculate the minimum circumcircle, obtain the target center point, and record the new color contour.

[0024] Step 2.4: Traverse all new color contour records and compare them with all old color contour records. If there is no old color contour, add it. If the distance between the center points of the new color contour and the old color contour is less than the threshold, they are considered to be the same and recorded.

[0025] Step 2.5: When all color contours have been recorded to a preset number, use the least squares method to calculate the optimal center point of the color contour;

[0026] Step 2.6: Manually input the custom world coordinate system coordinates corresponding to the color outline of each yellow spherical calibration object, and save the matching results.

[0027] In step 3, the camera's extrinsic data includes the camera's position and rotation vector in a custom real-world coordinate system.

[0028] Step 4, the triangulation positioning, specifically includes the following steps:

[0029] Step 4.1: In each video frame of each camera, identify and track the same yellow spherical marker, and calculate the optimal tracking point of the yellow spherical marker on each camera image based on multi-frame data;

[0030] Step 4.2: Perform distortion correction on the optimal tracking points from the three cameras to obtain points P1, P2, and P3;

[0031] Step 4.3: Perform triangulation on the points after distortion removal to obtain 3 triangulation coordinate points;

[0032] Step 4.4: Connect the three triangulation coordinate points obtained, and then use the least squares method to calculate the optimal coordinate point.

[0033] Compared with the prior art, the present invention has the following beneficial effects:

[0034] 1. This solution achieves high-precision 3D positioning under reasonable conditions through precise calibration of internal and external parameters and multi-camera perspective fusion calculation, effectively improving the accuracy and robustness of 3D positioning.

[0035] 2. Actual tests show that when the camera imaging resolution is high and the calibration accuracy is sufficient, the positioning error can reach the millimeter level, which can meet the high-precision positioning requirements of industrial inspection, precision assembly and robotic operations.

[0036] 3. Based on multi-camera vision, the target is captured by multiple cameras at different positions and angles. Combined with calibration parameters and triangulation principles, high positioning accuracy can be achieved without adding expensive hardware. Attached Figure Description

[0037] Figure 1 This is a schematic diagram of the equipment distribution setup for the triangulation and positioning system based on multi-camera vision according to the present invention.

[0038] Figure 2 This is a flowchart of the triangulation and positioning based on multi-camera vision according to the present invention. Detailed Implementation

[0039] The structure and working process of the present invention will be further described below with reference to the accompanying drawings.

[0040] The purpose of this invention is to propose a method for high-precision 3D positioning based on multi-camera vision. In view of the shortcomings of existing vision positioning methods in terms of accuracy and stability, this invention proposes a positioning scheme that combines precise calibration and multi-view fusion to improve the accuracy and robustness of 3D positioning.

[0041] A triangulation and positioning system based on multi-camera vision includes cameras, calibration objects, and tracking objects. The cameras consist of three cameras with large baseline differences but whose lenses face the same area. Multiple calibration objects are stationary at different locations within the overlapping fields of view of the three cameras, used for extrinsic parameter calibration. A single spherical tracking object is placed at any location within the overlapping field of view of the cameras for tracking and positioning. First, the camera intrinsic parameters are calibrated. Then, the camera extrinsic parameter data is obtained and calculated based on the calibration objects. Finally, triangulation and positioning are performed based on the camera intrinsic and extrinsic parameters.

[0042] Specific embodiments, such as Figure 1 , Figure 2 As shown:

[0043] A triangulation and positioning system based on multi-camera vision includes cameras, calibration objects, and tracking objects; among them, the cameras include three 4K cameras, and the baselines (relative displacement, rotation) between the cameras should be as large as possible, but their cameras should all be looking at the same area.

[0044] The calibration includes four yellow spherical calibration objects with a diameter of 4cm, which are placed statically within the overlapping areas of the three camera views. The accurate coordinates of each yellow spherical calibration object in the custom world coordinate system need to be obtained. The height of each yellow spherical calibration object in the custom world coordinate system is 0, which is used for camera extrinsic parameter calibration.

[0045] The tracking object is a yellow spherical tracker with a diameter of 4cm, placed at any location within the overlapping area of ​​the three camera views for tracking and positioning.

[0046] This embodiment mainly uses the OpenCV library for image processing and spatial positioning. The specific implementation process of the triangulation positioning method based on this system is as follows:

[0047] 1. Camera intrinsic parameter calibration, specifically including:

[0048] (1) Use a checkerboard pattern to calibrate the internal parameters of each camera.

[0049] (2) Place the chessboard pattern within the camera's field of view and capture images of the chessboard pattern at different positions and angles.

[0050] (3) Use the cv2.cvtColor function to convert the image to grayscale.

[0051] (4) Use the cv2.findChessboardCornersSB function to detect corner points for each frame of the image.

[0052] (5) Use cv2.cornerSubPix for subpixel precision.

[0053] (6) Use cv2.calibrateCamera to obtain the camera's intrinsic parameter matrix and distortion coefficients.

[0054] (7) Save the intrinsic parameter matrix and distortion coefficients for subsequent extrinsic parameter calculation and triangulation positioning.

[0055] 2. Acquire camera extrinsic data. In each camera's video frame, identify and track four yellow spherical calibration objects based on a color threshold. Calculate the optimal point (pixel coordinates) of each yellow spherical calibration object in the image based on multi-frame data. Then, manually input the coordinates of each yellow spherical calibration object in a custom world coordinate system, match and record the pixel coordinates with the coordinates in the custom world coordinate system, and finally obtain the data used for extrinsic calibration of each camera. Specifically, this includes:

[0056] (1) Convert the current frame image to HSV format using cv2.cvtColor.

[0057] (2) Use cv2.inRange to find the regions in the image that match the HSV range: H(5,30), S(77,255), V(160,255).

[0058] (3) Use cv2.findContours to find the target color contour.

[0059] (4) Traverse all color contours. If the area is greater than the threshold (set to 100 in this embodiment), calculate the minimum circumcircle, obtain the target center point (x,y), and record the new color contour.

[0060] (5) Traverse all new color contour records and compare them with all old color contour records. If there is no old color contour, add it. If the distance between the center point of the new color contour and the old color contour is less than the threshold (set to 10 in this embodiment), they are considered to be the same and are recorded.

[0061] (6) When all color contours have been recorded up to 100, the optimal color contour center point is calculated using the least squares method.

[0062] (7) Manually input the custom world coordinate system coordinates corresponding to the color outline of each yellow spherical calibration object, and match and save.

[0063] 3. Camera extrinsic parameter calculation: Using the pixel coordinates and real-world coordinates of the four yellow spherical calibration objects obtained in the previous step, calculate the extrinsic parameters for each camera, specifically including:

[0064] (1) Pass the corresponding pixel and real coordinates and camera intrinsic parameters into the cv2.solvePnP function to obtain the camera's extrinsic parameters (the camera's position and rotation vector in the custom real coordinate system).

[0065] (2) Save the extrinsic parameters of each camera for triangulation positioning.

[0066] 4. Triangulation and localization: In each video frame of each camera, the same yellow spherical marker is identified and tracked. Based on multi-frame data, the optimal tracking point (pixel coordinates) of the yellow spherical marker on each camera image is calculated. Based on the acquired intrinsic and extrinsic parameters of each camera, triangulation calculations are performed on the tracking point, and the final output is the 3D coordinates of the point in a custom world coordinate system. Specifically, this includes:

[0067] (1) Convert the current frame image to HSV format using cv2.cvtColor.

[0068] (2) Use cv2.inRange to find the regions in the image that match the HSV range: H(5,30), S(77,255), V(160,255).

[0069] (3) Use cv2.findContours to find the target color contour.

[0070] (4) Traverse all color contours. If the area is greater than 100, calculate the smallest circumcircle, obtain the target center point (x,y), and record the new color contour.

[0071] (5) Traverse all new color contour records and compare them with all old color contour records. If there is no old color contour, add it. If the distance between the center point of the new color contour and the old color contour is less than 10, they are considered to be the same and are recorded.

[0072] (6) When all color contours have been recorded to a total of 100, the optimal color contour center point (tracking point) is calculated using the least squares method.

[0073] (7) Use cv2.undistortPoints to remove distortion from the tracking points in the three cameras to obtain points P1, P2, and P3.

[0074] (8) Triangulate the points after distortion removal using cv2.triangulatePoints, P1-P2, P1-P3, and P2-P3 respectively. This will give you three triangulation coordinate points.

[0075] (9) Connect the three triangulation coordinate points obtained using np.vstack, and then use the least squares method to calculate the optimal coordinate point.

[0076] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0077] It should be understood that this solution is not limited to the specific embodiments described above. Devices and structures not described in detail herein should be understood as being implemented in a manner common to the art. Any person skilled in the art can make many possible variations and modifications to this solution, or modify it into equivalent embodiments, without departing from the scope of this solution, using the methods and techniques disclosed above. This does not affect the substantive content of this solution. Therefore, any simple modifications, equivalent changes, and alterations made to the above embodiments based on the technical essence of this solution, without departing from its scope, still fall within the protection scope of this solution.

Claims

1. A triangulation and positioning system based on multi-camera vision, characterized in that: The system includes cameras, calibration objects, and tracking objects. The cameras consist of three cameras with large baseline differences but whose lenses face the same area. Multiple calibration objects are stationary at different locations within the overlapping fields of view of the three cameras, used for extrinsic parameter calibration. A single spherical tracking object is placed at any location within the overlapping field of view of the cameras for tracking and positioning. First, the camera intrinsic parameters are calibrated. Then, the camera extrinsic parameter data is obtained and calculated based on the calibration objects. Finally, triangulation positioning is performed based on the camera intrinsic and extrinsic parameters.

2. The triangulation and positioning system based on multi-camera vision according to claim 1, characterized in that: The calibration objects consist of four yellow spherical calibration objects, each with a height of 0 in the custom world coordinate system.

3. The triangulation and positioning system based on multi-camera vision according to claim 1, characterized in that: The tracker is a yellow, spherical tracker.

4. The triangulation positioning method based on the system of claim 1, characterized in that: Includes the following steps: Step 1: Camera intrinsic parameter calibration, obtain and save the intrinsic parameter matrix and distortion coefficients for each camera; Step 2: Obtain camera extrinsic data. In each camera's video frame, identify and track all yellow spherical markers based on a color threshold. Calculate the optimal point of each yellow spherical marker in the image based on multi-frame data. Then, manually input the coordinates of each yellow spherical marker in a custom world coordinate system and match and record the pixel coordinates with the real coordinates in the custom world coordinate system. Step 3: Camera extrinsic parameter calculation. Using the pixel coordinates and real-world coordinates obtained in Step 2, calculate and save the extrinsic parameter data for each camera. Step 4: Triangulation and positioning. In each video frame of each camera, the same yellow spherical marker is identified and tracked. Based on the multi-frame data, the optimal tracking point of the yellow spherical marker on each camera image is calculated. Based on the intrinsic and extrinsic parameters of each camera, triangulation calculation is performed on the optimal tracking point, and finally the three-dimensional coordinates of the point in the custom world coordinate system are output.

5. The triangulation positioning method according to claim 4, characterized in that: In step 1, the intrinsic parameters of each camera are calibrated using a checkerboard pattern, which specifically includes the following steps: Step 1.1: Place the checkerboard pattern within the camera's field of view and capture images of the checkerboard pattern from different positions and angles; Step 1.2: Convert the acquired image to grayscale; Step 1.3: Detect corner points in each frame of the image and perform sub-pixel level precision; Step 1.4: Obtain the camera's intrinsic parameter matrix and distortion coefficients, and save them.

6. The triangulation positioning method according to claim 4, characterized in that: Step 2 specifically includes the following steps: Step 2.1: Convert the current frame image to HSV format; Step 2.2: Find the regions H(5,30), S(77,255), and V(160,255) in the image that match the HSV range, and find the target color contour; Step 2.3: Traverse all target color contours. For those that meet the area requirements, calculate the minimum circumcircle, obtain the target center point, and record the new color contour. Step 2.4: Traverse all new color contour records and compare them with all old color contour records. If there is no old color contour, add it. If the distance between the center points of the new color contour and the old color contour is less than the threshold, they are considered to be the same and recorded. Step 2.5: When all color contours have been recorded to a preset number, use the least squares method to calculate the optimal center point of the color contour; Step 2.6: Manually input the custom world coordinate system coordinates corresponding to the color outline of each yellow spherical calibration object, and save the matching results.

7. The triangulation positioning method according to claim 4, characterized in that: In step 3, the camera's extrinsic data includes the camera's position and rotation vector in a custom real-world coordinate system.

8. The triangulation positioning method according to claim 4, characterized in that: Step 4, the triangulation positioning, specifically includes the following steps: Step 4.1: In each video frame of each camera, identify and track the same yellow spherical marker, and calculate the optimal tracking point of the yellow spherical marker on each camera image based on multi-frame data; Step 4.2: Perform distortion correction on the optimal tracking points from the three cameras to obtain points P1, P2, and P3; Step 4.3: Perform triangulation on the points after distortion removal to obtain 3 triangulation coordinate points; Step 4.4: Connect the three triangulation coordinate points obtained, and then use the least squares method to calculate the optimal coordinate point.

Images