A variable baseline binocular camera target ranging system and method

Abstract

The application discloses a variable-baseline binocular camera target ranging system and method, and belongs to the technical field of computer vision and target ranging, which comprises a mechanical execution subsystem, an image acquisition subsystem and a data processing and control subsystem. The system dynamically adjusts the baseline length between a first camera and a second camera through a first mechanical arm and a second mechanical arm, collects a first group of images in a short baseline state, calculates a first distance estimation value and a first error variance, switches to a long baseline state to collect a second group of images, calculates a second distance estimation value and a second error variance, finally calculates a fusion coefficient based on a variance reciprocal weighting fusion algorithm, and performs weighted fusion on the two estimation values to obtain a final distance estimation value and a standard deviation. The application considers the near-distance matching reliability and the long-distance ranging precision, effectively suppresses random noise and eliminates the visual field blind area.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and target ranging technology, and in particular relates to a variable baseline binocular camera target ranging system and method. Background Technology

[0002] Binocular stereo vision technology simulates human eyes by using two cameras to capture the same scene from different angles. It calculates the depth information of the target based on the principle of parallax, with its core principle being triangulation. This technology has wide applications in fields such as autonomous driving, robot navigation, industrial inspection, and 3D reconstruction.

[0003] In traditional binocular vision ranging systems, the relative positions of the two cameras are usually fixed, meaning the baseline length is constant. According to the principles of triangulation, the accuracy of depth measurement is directly proportional to the baseline length and inversely proportional to the square of the target distance. This means: The contradiction of baseline length: shorter baselines are beneficial for feature matching of near-range targets (large overlapping field of view), but accuracy drops sharply at long distances; longer baselines, while improving the accuracy of long-distance measurements, are prone to blind spots for near-range targets and feature matching becomes more difficult (due to occlusion and excessive parallax). Fixed-baseline binocular systems struggle to simultaneously achieve high-precision measurements of both near and far scenes.

[0004] The impact of measurement noise: Due to limitations in the pixel discretization error of the camera sensor, changes in illumination, and matching errors in the image processing algorithm, the distance value calculated from a single static image often contains significant random noise.

[0005] Existing improvement schemes mostly focus on simple image algorithm optimization, while fewer employ dynamic baseline adjustment combined with time-series data fusion algorithms to comprehensively improve the robustness and accuracy of ranging. Therefore, it is particularly important to develop a ranging system that can dynamically adjust the baseline according to measurement needs and utilize observation data at different baseline lengths for filtering optimization. Summary of the Invention

[0006] The purpose of this invention is to provide a variable baseline binocular camera target ranging system and method, which resolves the contradiction between the difficulty of close-range feature matching and the low accuracy of long-range measurement in traditional fixed baseline binocular systems.

[0007] To achieve the above objectives, the present invention provides a variable baseline binocular camera target ranging system, comprising: a mechanical execution subsystem, an image acquisition subsystem, and a data processing and control subsystem; The mechanical execution subsystem includes a first robotic arm, a second robotic arm, and a base, with both the first and second robotic arms fixedly mounted on the base. The image acquisition subsystem includes a first camera and a second camera. The first camera is mounted on the end effector of the first robotic arm, and the second camera is mounted on the end effector of the second robotic arm. The data processing and control subsystem includes a control processor, a storage module, and an uncertainty assessment module. The control processor is electrically connected to the first robotic arm, the second robotic arm, the first camera, and the second camera, respectively, and is used to control the movement of the robotic arms to change the baseline length between the first camera and the second camera, and to execute image acquisition, feature matching, distance calculation, and data fusion algorithms. The uncertainty assessment module is configured to calculate the standard deviation of the final estimate based on the fused result.

[0008] Preferably, both the first and second robotic arms are multi-degree-of-freedom serial robotic arms, which can freely adjust the translation and rotation posture of the camera in three-dimensional space.

[0009] Preferably, the optical axes of the first camera and the second camera remain parallel in the initial state, and their imaging planes are coplanar.

[0010] This invention also provides a variable baseline binocular camera target ranging method, comprising the following steps: Step S1, system initialization and short baseline setting: controlling the first and second robotic arms to move to a preset initial observation pose, so that the baseline length between the first and second cameras is the first baseline length. ; Step S2, Initial Short Baseline Distance Measurement: While keeping the robotic arm stationary, the first and second cameras are simultaneously triggered to acquire the first left and first right images, respectively. Feature extraction and stereo matching are performed, and the first distance estimate is calculated based on the triangulation principle. and the first error variance ; Step S3, Baseline Dynamic Adjustment and Long Baseline Setting: Based on the first distance estimate Adaptively plan the motion trajectory of the robotic arm, control the first and / or second robotic arms to move to the secondary observation pose, and make the baseline length between the two cameras equal to the second baseline length. ,and ; Step S4, Long Baseline Secondary Distance Measurement: Under long baseline conditions, control the first and second cameras again to acquire the second left and second right images, perform feature extraction and stereo matching, and calculate the second distance estimate based on the triangulation principle. and the second error variance ; Step S5, Multi-scale data fusion and optimal estimation: Based on the first error variance Second error variance Calculate the fusion coefficient and use the fusion coefficient to estimate the first distance. Second distance estimate We perform weighted fusion to obtain the final distance estimate. And calculate the standard deviation of the final distance estimate. .

[0011] Preferably, in steps S2 and S4, the distance estimate is calculated based on the principle of triangulation. The calculation formula is as follows: ; in, Focal length Baseline length For parallax.

[0012] Preferably, in steps S2 and S4, feature extraction employs SIFT, SURF, or a deep learning-based feature matching algorithm.

[0013] Preferably, in step S3, the baseline dynamic adjustment and long baseline setting include the following steps: S31. Extremely close range exemption determination: If If the value is less than the preset safety threshold, the second baseline length is maintained. If the long baseline measurement is not performed, proceed to step S36; otherwise, proceed to step S32. S32. Calculate the upper limit of the overlapping field of view constraint. The expression is: ; in, For the camera's focal length, The physical width of the camera's image sensor. This is the threshold for field of view overlap rate. This is the first distance estimate; S33. Calculate the upper limit of the maximum disparity constraint. The expression is: ; in, To maximize search parallax; S34. Constraint Truncation: Calculate the baseline after constraint truncation. : ; ; in, This is the maximum baseline length allowed for the robotic arm. The minimum baseline length allowed for the robotic arm; S35. Accuracy compliance check and determination of the second baseline length: Given a target with desired ranging accuracy The expected standard deviation is then calculated using the following formula. : ; in, The standard deviation is used to estimate the disparity. like Then determine the length of the second baseline. And execute S36; like Then, the target with the desired ranging accuracy is re-entered to the control processor. If the target of the desired ranging accuracy is re-entered If the desired ranging accuracy target is not entered or is re-entered, then S35 will be executed again. Then take the current The system outputs the final distance estimate along with a warning, and simultaneously controls the first and second robotic arms to reset to their initial safe positions, thus terminating the ranging task. S36. Perform baseline changes: Calculate the amount of baseline change. Control the ends of the two robotic arms to move from their initial poses in opposite horizontal directions. After collision avoidance and field of view protection checks, the device moves to the secondary observation position, locks the joint, and triggers step S4.

[0014] Preferably, in step S5, the fusion coefficient The calculation formula is as follows: ; in, The first error variance, This represents the second error variance.

[0015] Preferably, in step S5, the final distance estimate is... The calculation formula is as follows: ; in, This is the first distance estimate. This is the second distance estimate.

[0016] Preferably, the standard deviation of the final distance estimate The calculation formula is as follows: .

[0017] Therefore, the present invention employs the above-described variable baseline binocular camera target ranging system and method, which has the following beneficial effects: (1) This invention dynamically adjusts the baseline by using a robotic arm. First, it uses a short baseline state to ensure large overlap of the near field of view and small ambiguity of feature matching. Then, it switches to a long baseline state to improve the sensitivity of long-distance measurement. Based on the weighted fusion algorithm, it outputs the optimal estimate, which effectively solves the technical problem that traditional fixed baseline binocular systems cannot simultaneously take into account high-precision measurement of near and far scenes.

[0018] (2) This invention does not rely on the results of a single static shooting. Instead, it uses two observation data, short baseline and long baseline, and uses linear minimum variance unbiased estimation based on the inverse variance weighting to fuse the data. This can effectively smooth the random noise of a single measurement, and the combined standard deviation after fusion is always less than the standard deviation of any independent measurement.

[0019] (3) The control processor of the present invention can adaptively plan the target value of the second baseline length based on the result of the preliminary distance measurement of the short baseline, i.e. the first distance estimate, so as to minimize the theoretical measurement error at the target distance and optimize the system response efficiency while ensuring accuracy.

[0020] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0021] Figure 1 This is a schematic diagram of the structure of a variable baseline binocular camera target ranging system according to the present invention; Figure 2 This is an overall flowchart of a variable baseline binocular camera target ranging method according to the present invention; Figure Labels 1. First robotic arm; 2. Second robotic arm; 3. First camera; 4. Second camera; 5. Base. Detailed Implementation

[0022] The following detailed description of embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0023] Example 1 Please see Figure 1 A variable baseline binocular camera target ranging system includes: a mechanical actuation subsystem, an image acquisition subsystem, and a data processing and control subsystem.

[0024] The mechanical execution subsystem includes a first robotic arm 1, a second robotic arm 2, and a base 5. Both the first robotic arm 1 and the second robotic arm 2 are fixedly mounted on the base 5, and are arranged side-by-side or at a certain angle according to the field of view requirements. In this embodiment, both the first robotic arm 1 and the second robotic arm 2 have high-precision repeatability. Both are multi-degree-of-freedom serial robotic arms, capable of freely adjusting the translation and rotation of the camera in three-dimensional space.

[0025] The image acquisition subsystem includes a first camera 3 and a second camera 4. The first camera 3 is mounted on the end effector of the first robotic arm 1, and the second camera 4 is mounted on the end effector of the second robotic arm 2. In this embodiment, the end effector is an end flange. Both the first camera 3 and the second camera 4 are high-resolution industrial cameras equipped with fixed-focus lenses. The optical axes of the first camera 3 and the second camera 4 are initially parallel, and their imaging planes are coplanar.

[0026] The data processing and control subsystem includes a control processor, a storage module, and an uncertainty assessment module. The control processor is electrically connected to the first robotic arm 1, the second robotic arm 2, the first camera 3, and the second camera 4, respectively, and is used to control the movement of the robotic arms to change the baseline length between the first camera 3 and the second camera 4, and to execute image acquisition, feature matching, distance calculation, and data fusion algorithms. The uncertainty assessment module is configured to calculate the standard deviation of the final estimate based on the fused results.

[0027] Example 2 Please see Figure 2 A target ranging method using a variable baseline binocular camera includes the following steps: Step S1, System Initialization and Short Baseline Setting: After the system is powered on, the control processor controls the first robotic arm 1 and the second robotic arm 2 to move to the preset initial observation pose, so that the baseline length between the first camera 3 and the second camera 4 is the first baseline length. The first baseline length Take the shorter value (first baseline length) The range of values ​​is In this embodiment, =15cm), ensuring that the field of view overlap area of ​​the first camera 3 and the second camera 4 is maximized and the feature point matching ambiguity is minimized under the first baseline length condition, thereby effectively reducing the blind zone of close-range measurement.

[0028] Step S2, Initial Short Baseline Ranging: While keeping the robotic arm stationary, the control processor synchronously triggers the first camera 3 and the second camera 4 to acquire the first left image and the first right image as the first set of images. The acquired first set of images is preprocessed, and feature points are extracted using SIFT and stereo matching is performed to obtain the first disparity. The first distance estimate is calculated based on the principle of triangulation. The calculation formula is as follows: ; in, Focal length The first baseline length, This is the first parallax.

[0029] In this embodiment, the lens focal lengths of the first camera 3 and the second camera 4 The pixel size is disparity obtained after stereo matching (Corresponding to 150 pixels), then the first distance estimate is: According to the first parallax The uncertainty is used to estimate the standard deviation of the first disparity estimate. Then estimate the first error variance. According to the error propagation formula: The first error variance was obtained. In the embodiment, a first parallax standard deviation is set. , pixels At this time, Because the baseline is short at this point, when the first distance estimate is... When it is large (e.g.) That is, for distant targets, the first error variance Possibly large, approximately However, for close-range targets, i.e., when the first distance estimate is... When it is small, the first error variance Smaller and more reliable.

[0030] Step S3, Baseline Dynamic Adjustment and Long Baseline Setting Steps: After completing the initial distance measurement, based on the first distance estimate... Adaptive planning of the robotic arm's motion trajectory; if the first distance estimate is... If the distance between the two cameras is too large, or to further improve accuracy, the system controls the first robotic arm 1 and / or the second robotic arm 2 to move horizontally in opposite directions, increasing the distance between the two cameras until they reach the secondary observation pose, and the baseline length between the two cameras is equal to the second baseline length. ,and During this process, the system monitors obstacle avoidance information in real time to ensure that the movement of the robotic arm does not obstruct the camera's view and that no collisions occur.

[0031] In this embodiment, the physical width of the image sensor Field of view overlap rate threshold Maximum search parallax (Corresponding to 256 pixels), the maximum allowable baseline length of the robotic arm Minimum baseline length allowed by the robotic arm Target ranging accuracy The standard deviation of the disparity estimation follows the method in step S2. .

[0032] S31. Extremely close range exemption determination: like Less than the preset safety threshold (in this embodiment) If the second baseline length is maintained, then the second baseline length is maintained. If the long baseline measurement is not performed, proceed to step S36; otherwise, proceed to step S32. In this example... Much greater than the safety threshold; S32. Calculate the upper limit of the overlapping field of view constraint. The expression is: ; Substituting the values ​​into the equation: ; S33. Calculate the upper limit of the maximum disparity constraint. The expression is: ; Substituting the values ​​into the equation: ; S34. Constraint Truncation: Calculate the baseline after constraint truncation. : ; ; in, This is the maximum baseline length allowed for the robotic arm. The minimum baseline length allowed for the robotic arm; S35. Accuracy compliance check and determination of the second baseline length: Given a target with desired ranging accuracy =0.01m, then the expected standard deviation is calculated using the following formula. : ; in, The standard deviation is used to estimate the disparity. because, Then determine the length of the second baseline. And execute S36; like Then, the target with the desired ranging accuracy is re-entered to the control processor. If the target of the desired ranging accuracy is re-entered If the desired ranging accuracy target is not entered or is re-entered, then S35 will be executed again. Then take the current The final distance estimate is output along with a warning, and the first robotic arm 1 and the second robotic arm 2 are reset to their initial safe positions to terminate the ranging task. S36. Perform baseline changes: Calculate the amount of baseline change. Control the ends of the two robotic arms to move from their initial poses in opposite horizontal directions. After collision avoidance and field of view protection checks, the device moves to the secondary observation position, locks the joint, and triggers step S4.

[0033] Step S4, Long Baseline Secondary Ranging: In the long baseline state, the control processor again controls the first camera 3 and the second camera 4 to trigger synchronously, acquiring the second left image and the second right image as the second set of images. The acquired second set of images is preprocessed, and the feature extraction and stereo matching process is repeated to obtain a new disparity. And calculate the second distance estimate based on the principle of triangulation. The calculation formula is as follows: ; in, Focal length The second baseline length, This is the second parallax; In this embodiment, the second disparity is obtained after long baseline ranging and stereo matching on the same target. (Corresponding to 144 pixels), then the second distance estimate is: According to the second parallax The uncertainty is used to estimate the standard deviation of the second disparity estimate. Then, estimate the second error variance. According to the error propagation formula: The second error variance is obtained. The standard deviation of the second parallax Pixels Under these conditions, we obtain the following: Due to the length of the second baseline The distance is relatively long. According to the principle of triangulation, under the same pixel error, the distance measurement error of distant targets is significantly reduced. When the first distance estimate is... When it is large, .

[0034] Step S5, Multi-scale data fusion and optimal estimation: Based on the first error variance Second error variance Calculate the fusion coefficient, where the fusion coefficient is... The calculation formula is as follows: ; This fusion coefficient This reflects the confidence weights of the two measurements; substituting this into the first error variance... Second error variance ,in, Calculated If the first measurement is more accurate ( ),but Approaching 0, the result mainly depends on If the second measurement is more accurate ( ),but Approaching 1, the result mainly depends on ; Calculate the final distance estimate using the corrected formula. Final distance estimate The calculation formula is as follows: ; in, This is the first distance estimate. This is the second distance estimate; Assume that the initial distance measurement of the short baseline in step S2 was obtained due to a large error. Substitute , , can be obtained This formula is essentially a linear minimum variance unbiased estimate based on variance, which can effectively smooth out random errors from a single measurement. The standard deviation of the final distance estimate is then calculated. As a confidence index, the standard deviation of the final distance from the estimated value The calculation formula is as follows: ; Substitute the first error variance Second error variance ,in ,get Output and The 073 control processor enables high-precision ranging of distant targets.

[0035] Therefore, this invention employs the aforementioned variable baseline binocular camera target ranging system and method. It utilizes a multi-degree-of-freedom tandem robotic arm to dynamically adjust the baseline length, and coordinates with two image acquisitions and disparity calculations by the binocular camera under both short and long baseline conditions. This generates two sets of observation data under different baseline lengths, thereby balancing near-range feature matching success rate and long-range ranging accuracy across the entire measurement range. It effectively suppresses random noise in single measurements and eliminates blind spots at specific angles. Simultaneously, this invention utilizes a multi-scale data fusion algorithm based on inverse variance weighting, combined with an uncertainty assessment module, to calculate the final distance estimate and its standard deviation as output results, thus achieving accurate estimation of target distance and quantitative error assessment.

[0036] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the technical solutions of the present invention, and these modifications or equivalent substitutions cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. A variable baseline binocular camera target ranging system, characterized in that, include: Mechanical actuation subsystem, image acquisition subsystem, and data processing and control subsystem; The mechanical execution subsystem includes a first robotic arm (1), a second robotic arm (2), and a base (5). The first robotic arm (1) and the second robotic arm (2) are both fixedly installed on the base (5). The image acquisition subsystem includes a first camera (3) and a second camera (4). The first camera (3) is mounted on the end effector of the first robotic arm (1), and the second camera (4) is mounted on the end effector of the second robotic arm (2). The data processing and control subsystem includes a control processor, a storage module, and an uncertainty assessment module. The control processor is electrically connected to the first robotic arm (1), the second robotic arm (2), the first camera (3), and the second camera (4), respectively, and is used to control the movement of the robotic arms to change the baseline length between the first camera (3) and the second camera (4), and to execute image acquisition, feature matching, distance calculation, and data fusion algorithms. The uncertainty assessment module is configured to calculate the standard deviation of the final estimation result based on the fused result.

2. The variable baseline binocular camera target ranging system according to claim 1, characterized in that: The first robotic arm (1) and the second robotic arm (2) are both multi-degree-of-freedom serial robotic arms, which can freely adjust the translation and rotation posture of the camera in three-dimensional space.

3. The variable baseline binocular camera target ranging system according to claim 1, characterized in that: The optical axes of the first camera (3) and the second camera (4) remain parallel in the initial state, and their imaging planes are coplanar.

4. A variable baseline binocular camera target ranging method, using a variable baseline binocular camera target ranging system as described in any one of claims 1-3, characterized in that, Includes the following steps: Step S1, System Initialization and Short Baseline Setting: Control the first robotic arm (1) and the second robotic arm (2) to move to the preset initial observation pose, so that the baseline length between the first camera (3) and the second camera (4) is the first baseline length. ; Step S2, Initial short baseline ranging: While keeping the robotic arm stationary, the first camera (3) and the second camera (4) are simultaneously triggered to acquire the first left image and the first right image, perform feature extraction and stereo matching, and calculate the first distance estimate based on the triangulation principle. and the first error variance ; Step S3, Baseline Dynamic Adjustment and Long Baseline Setting: Based on the first distance estimate Adaptively plan the motion trajectory of the robotic arm, control the first robotic arm (1) and / or the second robotic arm (2) to move to the secondary observation pose, so that the baseline length between the two cameras is the second baseline length. ,and ; Step S4, Long Baseline Secondary Distance Measurement: Under the long baseline state, control the first camera (3) and the second camera (4) again to acquire the second left image and the second right image, perform feature extraction and stereo matching, and calculate the second distance estimate based on the triangulation principle. and the second error variance ; Step S5, Multi-scale data fusion and optimal estimation: Based on the first error variance Second error variance Calculate the fusion coefficient and use the fusion coefficient to estimate the first distance. Second distance estimate We perform weighted fusion to obtain the final distance estimate. And calculate the final distance estimate. Standard deviation .

5. The variable baseline binocular camera target ranging method according to claim 4, characterized in that: In steps S2 and S4, distance estimates are calculated based on the principle of triangulation. The calculation formula is as follows: ； in, For the camera's focal length, Baseline length For parallax.

6. The variable baseline binocular camera target ranging method according to claim 5, characterized in that, In steps S2 and S4, feature extraction employs SIFT, SURF, or deep learning-based feature matching algorithms.

7. The variable baseline binocular camera target ranging method according to claim 6, characterized in that, Step S3, baseline dynamic adjustment and long baseline setting includes the following steps: S31. Extremely close range exemption determination: If If the value is less than the preset safety threshold, the second baseline length is maintained. If the long baseline measurement is not performed, proceed to step S36; otherwise, proceed to step S32. S32. Calculate the upper limit of the overlapping field of view constraint. The expression is: ； in, For the camera's focal length, The physical width of the camera's image sensor. This is the threshold for field of view overlap rate. This is the first distance estimate; S33. Calculate the upper limit of the maximum disparity constraint. The expression is: ； in, To maximize search parallax; S34. Constraint Truncation: Calculate the baseline after constraint truncation. : ； ； in, This is the maximum baseline length allowed for the robotic arm. The minimum baseline length allowed for the robotic arm; S35. Accuracy compliance check and determination of the second baseline length: Given a target with desired ranging accuracy The expected standard deviation is then calculated using the following formula. : ； in, The standard deviation is used to estimate the disparity. like Then determine the length of the second baseline. And execute S36; like Then, the target with the desired ranging accuracy is re-entered to the control processor. If the target of the desired ranging accuracy is re-entered If the desired ranging accuracy target is not entered or is re-entered, then S35 will be executed again. Then take the current The final distance estimate is output along with a warning, and the first robotic arm (1) and the second robotic arm (2) are reset to their initial safe positions to terminate the distance measurement task. S36. Perform baseline changes: Calculate the amount of baseline change. Control the ends of the two robotic arms to move from their initial poses in opposite horizontal directions. After collision avoidance and field of view protection checks, the device moves to the secondary observation position, locks the joint, and triggers step S4.

8. The variable baseline binocular camera target ranging method according to claim 7, characterized in that, In step S5, the fusion coefficient The calculation formula is as follows: ； in, The first error variance, This represents the second error variance.

9. A target ranging method using a variable baseline binocular camera according to claim 8, characterized in that, In step S5, the final distance estimate The calculation formula is as follows: ； in, This is the first distance estimate. This is the second distance estimate.

10. A target ranging method using a variable baseline binocular camera according to claim 9, characterized in that, Standard deviation of the final distance estimate The calculation formula is as follows: 。

Images