Long-distance ranging method based on binocular vision
By improving the YOLOv11 model and combining attention, coarse matching, and fine matching modules, the problem of accurate localization and precision in long-distance target detection was solved, achieving efficient and low-cost sub-pixel accuracy ranging.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- SOUTH CHINA UNIV OF TECH
- Filing Date
- 2026-02-11
- Publication Date
- 2026-06-02
AI Technical Summary
Existing technologies have bottlenecks in the accurate positioning and identification of distant targets and the accuracy of feature point detection, especially in the detection of distant targets where performance degrades, computational costs are high, and it is difficult to achieve high-precision ranging.
An improved YOLOv11 model is adopted, which combines an attention module, a coarse matching module, and a fine matching module. The feature map information is enhanced by horizontal and vertical pooling. Coarse matching is performed using Census transform and cost aggregation, and fine matching is performed by constructing a continuous implicit function, thereby achieving sub-pixel accuracy ranging.
It improves the accuracy and robustness of long-range target detection, reduces computational costs, adapts to different scenarios and tasks, and achieves high-precision end-to-end ranging.
Smart Images
Figure CN122134824A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the technical fields of computer vision, deep learning, and 3D measurement, and in particular to a long-distance ranging method based on binocular vision. Background Technology
[0002] Binocular stereo vision technology, by simulating the principles of human vision, can acquire three-dimensional depth information of a scene, thus showing broad application prospects in fields such as autonomous driving and robot navigation. However, in practical applications involving high-precision ranging of long-distance, specific targets, existing technologies face the following two core bottlenecks, specifically:
[0003] The first bottleneck lies in the accurate localization and recognition of distant targets. Traditional target detection algorithms and standard YOLO models experience a sharp performance drop when handling distant targets. Their shortcomings are mainly reflected in: 1. Distant pedestrians occupy only a very small number of pixels in an image, their features are weak and easily affected by background noise; 2. While downsampling in standard convolutional neural networks can expand the receptive field, it loses spatial location information for small targets; 3. Existing attention mechanisms, such as CBAM, also blur the precise coordinates of small targets during feature enhancement through global pooling, hindering accurate localization; 4. Standard non-maximum suppression methods, when dealing with dense or occluded targets, easily suppress correct detection boxes, leading to localization errors.
[0004] The second bottleneck lies in the accuracy limit of feature point detection. Although Neural Radiation Field (NeRF) models can represent scenes as continuous implicit functions to overcome the accuracy limitations of discrete pixels, existing general-purpose NeRF models typically rely on multi-view images as input and aim for global reconstruction of the entire scene. This results in slow training convergence, high computational costs, and difficulty in adapting to ranging scenarios with only binocular image input and extremely high real-time requirements. Furthermore, traditional NeRF models focus on visual rendering quality rather than precise measurement of geometric surfaces. Without prior constraints on local spatial geometry, directly using them to calculate depth for distant targets is prone to overfitting or depth drift, failing to meet the need for real-time, sub-pixel-level high-precision ranging for specific target regions.
[0005] In summary, how to simultaneously address the bottlenecks in feature extraction, model training, and post-processing is a technical challenge that urgently needs to be overcome in this field. Summary of the Invention
[0006] The purpose of this invention is to overcome the shortcomings and deficiencies of the prior art and to propose a long-distance ranging method based on binocular vision, which can realize rapid and accurate detection of long-distance targets, reduce working costs, and improve detection efficiency and accuracy.
[0007] To achieve the above objectives, the technical solution provided by this invention is as follows: a long-distance ranging method based on binocular vision. This method utilizes an improved YOLOv11 model to achieve long-distance ranging. The improvement of the improved YOLOv11 model is the addition of an attention module, a coarse matching module, and a fine matching module to the original YOLOv11 model. The attention module performs horizontal and vertical pooling on the feature map to fuse and output image information, enhancing the improved YOLOv11 model's ability to accurately locate distant targets. The coarse matching module receives the image information output by the attention module, performs encoding transformation on the pixels of the feature map to generate encoding groups, and outputs discrete pixel pairs in integer pixel units, providing an initial basis for fine matching. The fine matching module iteratively trains the discrete pixel pairs obtained by the coarse matching module through multiple rounds of training, constructs the scene geometry into a continuous implicit function, and transforms the discrete pixel pairs into continuous pixel regions by numerically analyzing this continuous implicit function, extracting depth values to achieve sub-pixel accuracy ranging.
[0008] The specific implementation of the long-distance ranging method includes the following steps:
[0009] Collect and preprocess pedestrian images from pre-detection to obtain corrected pedestrian images;
[0010] The corrected pedestrian images are processed using a pre-trained improved YOLOv11 model as follows:
[0011] The pedestrian image is input into the backbone network of the improved YOLOv11 model for feature extraction, and the original feature map is output.
[0012] The original feature map is transmitted to the attention module, which performs horizontal and vertical pooling and weighting on the original feature map to generate an enhanced feature map, which is then transmitted to the detection head of the improved YOLOv11 model.
[0013] The detection head outputs a series of target detection boxes based on the enhanced feature map, and uses the weighted box fusion algorithm in the attention module to fuse the series of target detection boxes to output the final two-dimensional bounding box.
[0014] The coarse matching module receives the two-dimensional bounding box, and extracts the corresponding core region in the left and right views after stereo correction based on the two-dimensional bounding box. It then performs feature point matching and cost aggregation on the core region and outputs a series of discrete pixel pairs.
[0015] The fine matching module constructs a local 3D bounding space containing the target object by combining a series of discrete pixel pairs obtained by the coarse matching module and the 2D bounding box obtained by the attention module. The truncated signed distance function is used as the geometric representation to construct the scene geometry within the local 3D bounding space as a continuous implicit function. Through real-time supervised training on the local 3D bounding space, a series of discrete pixel pairs are fitted and transformed into a continuous 3D surface. The spatial intersection of this continuous 3D surface and the line of sight is solved iteratively along the line of sight. The depth value of the spatial intersection is aggregated and decided to output the final ranging result.
[0016] Furthermore, the collected pedestrian images undergo preprocessing, including the following steps:
[0017] Data Acquisition: A large number of multi-pose chessboard images were acquired using a binocular camera;
[0018] Intrinsic parameter calibration: Using the chessboard image, single-target calibration is performed on the left and right cameras respectively, and the intrinsic parameters of the left and right cameras are calculated.
[0019] Extrinsic parameter calibration: Based on the intrinsic parameters, the binocular camera is calibrated to obtain the rotation and translation matrix describing the relative positional relationship between the left and right cameras, i.e., the extrinsic parameters;
[0020] Epipolar correction: Based on the intrinsic and extrinsic parameters, calculate the stereo correction transformation matrix so that the corrected left and right views satisfy the constraint that the epipolar lines are parallel and aligned.
[0021] Stereo correction: Acquire pre-detected pedestrian images, use the acquired intrinsic parameters, extrinsic parameters and stereo correction transformation matrix to perform stereo correction on the input pedestrian images, record and output the corrected pedestrian images.
[0022] Furthermore, the attention module is mainly composed of coordinate attention (CA) units responsible for filtering, and is configured with a dynamic non-monotonic focusing (WIoU) loss function for training optimization, and a weighted bounding box fusion (WBF) algorithm for output fusion.
[0023] The coarse matching module consists of a Census transformation method responsible for localization and a cost aggregation unit based on the principle of regional consistency.
[0024] The fine matching module consists of a NeRF model of neural radiation field characterized by the truncated symbolic distance function TSDF and a ray stepping solver, wherein the NeRF model of neural radiation field is a specific implementation of a continuous implicit function and the ray stepping solver is a specific implementation of iterative solution.
[0025] Furthermore, the attention module uses the coordinate attention (CA) unit to perform global average pooling in the horizontal and vertical directions on the corrected pedestrian image to generate feature maps with position awareness and orientation information.
[0026] The dynamic non-monotonic focusing WIoU loss function records a large number of predicted boxes containing pedestrian target positions and categories based on the generated feature map with position awareness and orientation information during the model training phase. By calculating the confidence of the predicted boxes, the weight of simple samples is dynamically reduced, and a series of target detection boxes and their confidence scores are output.
[0027] In the detection output stage, the weighted bounding box (WBF) algorithm clusters a series of target detection boxes according to their spatial location and intersection-union ratio (IU). For each cluster, the boundary coordinates of the target detection boxes are weighted and averaged according to their confidence scores. The target detection boxes are then fused, and the coordinates and confidence scores of the fused target detection boxes are calculated. All target detection boxes are then fused into the final two-dimensional bounding box.
[0028] Furthermore, the coarse matching module receives the two-dimensional bounding box obtained by the attention module, segments the target and background in the corrected left and right views, extracts the core region, and uses the core region as input to the Census transform method for calculation and encoding; a core region mask matrix is defined as:
[0029] ;
[0030] Based on the aforementioned core region mask matrix, four center points and four support points within their neighborhoods are selected. Census transform binary encoding is performed on the center points and the four support points to generate a coding group containing five sets of codes. Coding group A and coding group B are generated for the left and right views, respectively. To calculate the initial matching cost, for each pixel to be matched, Census transform binary encoding is iterated multiple times within a preset disparity search range. In each iteration, for a preset disparity value, the coding values of coding group A and coding group B are compared bit by bit, and the number of all different bits is summed. This summation value is recorded as the Hamming distance value. The Hamming distance value of each coding group is recorded as the initial matching cost, and all initial matching costs are then constructed into an initial cost volume. The aforementioned core region mask matrix... The number is a random number representing the pixel value. The subscripts 1 to 21 indicate the position index numbers of the pixels involved in the calculation, arranged in order. The four support points are selected from the four pixels above, below, left, and right of the center point.
[0031] The cost aggregation unit based on the regional consistency principle adopts a preset aggregation window. Based on the initial cost body, it accumulates and sums the initial matching costs of all pixels in the window under the same disparity, transforms the initial matching costs into aggregated matching costs, and thus constructs an aggregated cost body from all aggregated matching costs.
[0032] Finally, iterating through the aggregation cost volume, for each pixel in the left and right views, the integer disparity that minimizes its aggregation matching cost is selected and denoted as the optimal matching disparity for that pixel. , where i represents the i-th pixel; calculate the minimum aggregate matching cost for all pixels, obtain the best matching disparity for each pixel, convert the disparity of each pixel into a depth value, and output a series of sparse discrete coarse matching point pairs, i.e., a series of discrete pixel pairs, where the disparity-depth conversion process is represented as:
[0033] ;
[0034] In the formula, represents the Z-axis coordinate of the pixel, b represents the baseline length of the stereo camera, f is the focal length of the stereo camera, and d represents the disparity of the pixel.
[0035] Furthermore, the fine matching module processes the series of discrete pixel pairs obtained by the coarse matching module and the two-dimensional bounding box obtained by the attention module, including the following steps:
[0036] For all input discrete pixel pairs Processing is carried out, among which The coordinates of the point are the X-axis coordinates. The coordinates are the Y-axis coordinates of the points, with subscript l indicating the left view and subscript r indicating the right view; extract the best matching disparity for each pair of points. Construct a disparity sample set; calculate the median of this disparity sample set, denoted as the central disparity. Where the subscript median indicates the median of the statistics; the central disparity Convert to scene center depth The subscript "center" indicates the location of the scene center; the depth of the scene center is... It will serve as the benchmark for constructing a local three-dimensional space;
[0037] Based on center depth Extend the preset buffer distance Determine the near plane and far plane Wherein, the subscript near represents the near plane boundary of the three-dimensional space, and the subscript far represents the far plane boundary of the three-dimensional space; the Z-axis boundary of the three-dimensional space is defined; using the intrinsic parameters of the binocular camera, the four corner points of the two-dimensional bounding box are back-projected onto... and On these two depth planes, eight vertices in three-dimensional space are obtained; by calculating the maximum and minimum coordinates of the eight vertices on the X and Y axes, and combining them with the defined depth range, a minimum three-dimensional bounding space is constructed.
[0038] The minimum three-dimensional bounding space is projected back onto the stereo-corrected left and right view images, respectively, and dense image patches containing complete visual information of the local scene are cropped out. Using these dense image patches as supervision information, and employing the truncated signed distance function (TSDF) as the internal geometric representation, a neural radiation field (NeRF) model is constructed to concretize the continuous implicit function. This NeRF model is then subjected to instantaneous iterative training within the minimum three-dimensional bounding space, and the resulting continuous implicit function is denoted as... The NeRF model of neural radiation field uses three-dimensional floating-point coordinates. As a continuous implicit function Input, output three-dimensional floating-point coordinates The truncation distance value to the nearest object surface is denoted as the TSDF value; this truncation distance value is obtained by considering all objects that meet the criteria. The surface constructed from the points is defined as a three-dimensional continuous surface, providing a continuous geometric representation for subsequent numerical solutions; among which, In this context, x represents the X-axis coordinate value, and y represents the Y-axis coordinate value. This represents the Z-axis coordinate value.
[0039] Furthermore, the fine matching module utilizes the trained NeRF model of neural radiation field and a three-dimensional continuous surface, employing a ray stepping solver as the specific implementation of the iterative solution along the line of sight, with the optical center of the left camera as the reference point. The origin is defined as follows: where the subscript `left` indicates the left camera of the stereo camera; the coordinates of the two-dimensional pixel point in the image that requires depth query are denoted as... u represents the X-axis coordinate of a two-dimensional pixel, and v represents the Y-axis coordinate of a two-dimensional pixel; using the intrinsic parameters of a stereo camera, this two-dimensional pixel coordinate... Perform a back projection transformation and calculate from The direction vector D to the center of this two-dimensional pixel is the ray direction; with the left camera optical center... Together with the direction vector D, a three-dimensional spatial ray parameterized by parameter t is defined. :
[0040] ;
[0041] To find the 3D intersection point of the TSDF value convergence minimum threshold, points on the ray close to the camera are selected. Begin iterative stepping, where, The current iteration step center point, The current index represents the current position in the current iteration step, and the next index represents the predicted position in the next iteration step. The iterative calculation steps are as follows:
[0042] 1) Query The TSDF value in the NeRF model of neural radiation field is obtained by denoting the truncated symbol distance as s:
[0043] ;
[0044] 2) Proceed a distance s along the ray direction D, and predict the center point of the next iteration step. The location is shown below:
[0045] ;
[0046] 3) Repeat steps 1) and 2) until s converges to the minimum threshold, and record the results. 3D intersection The subscript `final` indicates the final intersection point that satisfies the convergence condition; this 3D intersection point is recorded. The Z-axis coordinate value of the three-dimensional intersection point The Z-axis coordinate value is the desired sub-pixel precision depth value, and a sample set consisting of sub-pixel precision depth values is generated.
[0047] 4) The sample set consisting of sub-pixel precision depth values resolved by the NeRF model of neural radiation field is compiled into a depth value list. The depth value list is aggregated and decided by a median voting mechanism. The median value selected by the aggregation decision is recorded as the final depth value. The final depth value is output and recorded as the final long-distance measurement result.
[0048] Compared with the prior art, the present invention has the following advantages and beneficial effects:
[0049] 1. Compared with other long-distance ranging methods, this invention improves robustness while maintaining detection accuracy. The proposed method, by setting an attention module and introducing coordinate attention (CA) units, can accurately acquire the position information of small targets at a distance, compensating for the spatial information loss caused by pooling operations in traditional convolutional pooling. During the training phase, a dynamic non-monotonic focusing WIoU loss function is configured to reduce the weight of simple samples and focus on the regression of high-quality anchor boxes, accelerating model convergence and improving generalization ability. In the output phase, a weighted box fusion WBF algorithm is used to replace the traditional non-maximum suppression method, effectively utilizing the redundant information of overlapping candidate boxes, significantly improving the positioning accuracy of the final two-dimensional bounding box, and providing a high-precision region of interest for subsequent ranging.
[0050] 2. In long-distance ranging, this invention combines a coarse matching module with the Census transform method and a cost aggregation unit based on the principle of region consistency to mine robust geometric features, reduce feature interference in long-distance and weak texture environments, provide discrete pixel-level three-dimensional space construction, and effectively guide the search process of the subsequent fine matching module.
[0051] 3. This invention sets up a fine matching module to address the problem of large discrete pixel disparity error in long-distance ranging. By constructing a neural radiation field NeRF model characterized by the truncated symbolic distance function TSDF, the discrete truncated symbolic distance values are fitted and transformed into a continuous three-dimensional surface expression, which improves the mapping ability from discrete space to continuous space, thereby improving the iterative solution capability to obtain spatial intersection points with sub-pixel accuracy.
[0052] 4. The method of the present invention has a wide range of applications in computer vision tasks, can realize end-to-end training, has strong adaptability to different scenarios and tasks, and has broad application prospects. Attached Figure Description
[0053] Figure 1 This is a flowchart illustrating the method of the present invention.
[0054] Figure 2 This is a schematic diagram of polar line correction in this embodiment.
[0055] Figure 3 The diagram shows the structure of the improved YOLOv11 model. In the diagram, Backbone represents the backbone network, Neck represents the neck network, Head represents the detection head, Conv represents the convolutional unit, C3k2 represents the cross-stage local feature extraction module, SPPF represents the fast spatial pyramid pooling module, C2PSA represents the cross-stage local module with spatial attention, CA represents the coordinate attention unit, Upsample represents the upsampling layer, Concat represents the concatenation layer, and Detect represents the detection layer.
[0056] Figure 4 This is a schematic diagram of the coarse matching module in this embodiment; in the diagram, Hamming represents the Hamming distance value.
[0057] Figure 5 This is a schematic diagram illustrating the principle of ray stepping depth calculation in the NeRF model of the neural radiation field in this embodiment. Detailed Implementation
[0058] The present invention will be further described in detail below with reference to the embodiments and accompanying drawings, but the embodiments of the present invention are not limited thereto.
[0059] like Figure 1 As shown, this embodiment discloses a long-distance ranging method based on binocular vision. This method utilizes an improved YOLOv11 model to achieve long-distance ranging. The improvement of the improved YOLOv11 model is the addition of an attention module, a coarse matching module, and a fine matching module to the original YOLOv11 model. The attention module performs horizontal and vertical pooling on the feature map to fuse and output image information, enhancing the improved YOLOv11 model's ability to accurately locate distant targets. The coarse matching module receives the image information output by the attention module, performs encoding transformation on the pixels of the feature map to generate encoding groups, and outputs discrete pixel pairs in integer pixel units, providing an initial basis for fine matching. The fine matching module iteratively trains the discrete pixel pairs obtained by the coarse matching module through multiple rounds, constructs the scene geometry into a continuous implicit function, and transforms the discrete pixel pairs into continuous pixel regions by numerically analyzing this continuous implicit function, extracting depth values to achieve sub-pixel accuracy ranging.
[0060] The specific implementation of the long-distance ranging method includes the following steps:
[0061] 1) Acquire pre-detected pedestrian images and perform preprocessing to obtain corrected pedestrian images, as follows:
[0062] A large number of checkerboard images with multiple poses were acquired using a binocular camera. Using these checkerboard images, single-target calibration was performed on the left and right cameras respectively, and the intrinsic parameters of the left and right cameras were calculated. Based on the intrinsic parameters, binocular calibration was performed on the binocular cameras, and the rotation and translation matrices describing the relative positional relationship between the left and right cameras, i.e., the extrinsic parameters, were calculated. Based on the intrinsic and extrinsic parameters, a stereo correction transformation matrix was calculated to ensure that the corrected left and right views satisfy the constraints of parallel epipolar lines and row alignment, such as... Figure 2 As shown, the pre-detected pedestrian images are acquired, and the obtained intrinsic parameters, extrinsic parameters, and stereo correction transformation matrix are used to perform stereo correction on the input pedestrian images. The corrected pedestrian images are then recorded and output.
[0063] 2) The corrected pedestrian images are processed using the pre-trained improved YOLOv11 model as follows:
[0064] The pedestrian image is input into the backbone network of the original YOLOv11 model for feature extraction, outputting a raw feature map; the raw feature map is then internally transmitted to the attention module, such as... Figure 3 As shown, a coordinate attention (CA) unit is inserted after the C2PSA layer in the backbone network of the original YOLOv11 model. This CA unit is located in the 12th layer of the network and connects the upsampling layer (Upsample) in the 13th layer and the concatenation layer (Concat) in the 21st layer, forming a new network structure. The new network structure receives the original feature map through the coordinate attention (CA) unit and performs global average pooling in the horizontal and vertical directions on the corrected pedestrian image to generate feature maps with position awareness and orientation information.
[0065] The dynamic non-monotonic focusing WIoU loss function records a large number of predicted boxes containing pedestrian target positions and categories based on the generated feature map with position awareness and orientation information during the model training phase. By calculating the confidence of the predicted boxes, the weight of simple samples is dynamically reduced, and a series of target detection boxes and their confidence scores are output.
[0066] In the detection output stage, the weighted bounding box (WBF) algorithm clusters a series of target detection boxes according to their spatial location and intersection-union ratio (IU). For each cluster, the boundary coordinates of the target detection boxes are weighted and averaged according to their confidence scores. The target detection boxes are then fused, and the coordinates and confidence scores of the fused target detection boxes are calculated. All target detection boxes are then fused into the final two-dimensional bounding box.
[0067] The coarse matching module receives the two-dimensional bounding box obtained by the attention module, segments the target and background in the corrected left and right views, extracts the core region, and uses the core region as input to the Census transform method for calculation and encoding; a core region mask matrix is defined as:
[0068] ;
[0069] Based on the aforementioned core region mask matrix, four center points and four support points within their neighborhoods are selected. Census transform binary encoding is performed on the center points and the four support points to generate a coding group containing five sets of codes. Coding group A and coding group B are generated for the left and right views, respectively. To calculate the initial matching cost, for each pixel to be matched, Census transform binary encoding is iterated multiple times within a preset disparity search range. In each iteration, for a preset disparity value, the coding values of coding group A and coding group B are compared bit by bit, and the number of all different bits is summed. This summation is recorded as the Hamming distance value. The Hamming distance value of each coding group is recorded as the initial matching cost, and all initial matching costs are then constructed into an initial cost volume. The aforementioned core region mask matrix... The number is a random number representing the pixel value. The subscripts 1 to 21 indicate the position index numbers of the pixels involved in the calculation, arranged in order. The four support points are selected from the four pixels above, below, left, and right of the center point.
[0070] like Figure 4 As shown in the left figure Matching a pixel in the right image The corresponding point, the pixel coordinates in the left image are Where u is the X-axis coordinate and v is the Y-axis coordinate; according to stereo calibration, corresponding points in the left and right views lie on the same epipolar line, and the X-axis coordinates of corresponding points differ by a disparity value. Therefore, the coordinates of the corresponding points in the right view are... Where ud is the X-axis coordinate and d is the parallax. (The coordinates of the points in the left figure are used as examples.) Using the center point as the reference point, select a 5×5 rectangular window with a center pixel value of 15. Compare each neighboring pixel value with the center value of 15. If a neighboring pixel value is greater than the center pixel value, record it as 1; if it is less than or equal to the center pixel value, record it as 0. Aggregate the above results to form a 24-bit string: 110011010010111011110000, denoted as code group A. Select coordinate points in the right figure. Centered on a point with a value of 52, a 5×5 rectangular window is selected. Each neighboring pixel within the window is compared with the center value of 52, generating another 24-bit bit string: 110011010010111011110011, denoted as code group B. Based on this, the Hamming distance is calculated. The number of different characters at corresponding positions in the two code groups is calculated to be 2, which is used as the initial matching cost. All initial matching costs are then used to construct the initial cost body.
[0071] The cost aggregation unit based on the regional consistency principle adopts a preset aggregation window. Based on the initial cost body, it accumulates and sums the initial matching costs of all pixels in the window under the same disparity, transforms the initial matching costs into aggregated matching costs, and thus constructs an aggregated cost body from all aggregated matching costs.
[0072] Finally, iterating through the aggregation cost volume, for each pixel in the left and right views, the integer disparity that minimizes its aggregation matching cost is selected and denoted as the optimal matching disparity for that pixel. Where the subscript i represents the i-th pixel; calculate the minimum cost of aggregate matching for all pixels, obtain the best matching disparity for each pixel, convert the disparity of each pixel into a depth value, and output a series of sparse discrete coarse matching point pairs, i.e., a series of discrete pixel pairs. The disparity-depth conversion process is represented as follows:
[0073] ;
[0074] In the formula, represents the Z-axis coordinate of the pixel, b represents the baseline length of the stereo camera, f is the focal length of the stereo camera, and d represents the disparity of the pixel.
[0075] The fine matching module processes a series of discrete pixel pairs obtained by the coarse matching module and the two-dimensional bounding box obtained by the attention module, including the following steps:
[0076] For all input discrete pixel pairs Processing is carried out, among which The coordinates of the point are the X-axis coordinates. The coordinates are the Y-axis coordinates of the points, with subscript l indicating the left view and subscript r indicating the right view; extract the best matching disparity for each pair of points. Construct a disparity sample set; calculate the median of this disparity sample set, denoted as the central disparity. Where the subscript median indicates the median of the statistics; the central disparity Convert to scene center depth The subscript "center" indicates the location of the scene center; the depth of the scene center is... It will serve as the benchmark for constructing a local three-dimensional space;
[0077] Based on center depth Extend the preset buffer distance Determine the near plane and far plane Wherein, the subscript near represents the near plane boundary of the three-dimensional space, and the subscript far represents the far plane boundary of the three-dimensional space; the Z-axis boundary of the three-dimensional space is defined; using the intrinsic parameters of the binocular camera, the four corner points of the two-dimensional bounding box are back-projected onto... and On these two depth planes, eight vertices in three-dimensional space are obtained; by calculating the maximum and minimum coordinates of the eight vertices on the X and Y axes, and combining them with the defined depth range, a minimum three-dimensional bounding space is constructed.
[0078] The minimum three-dimensional bounding space is projected back onto the stereo-corrected left and right view images, respectively, and dense image patches containing complete visual information of the local scene are cropped out. Using these dense image patches as supervision information, and employing the truncated signed distance function (TSDF) as the internal geometric representation, a neural radiation field (NeRF) model is constructed to concretize the continuous implicit function. This NeRF model is then subjected to instantaneous iterative training within the minimum three-dimensional bounding space, and the resulting continuous implicit function is denoted as... The NeRF model of neural radiation field uses three-dimensional floating-point coordinates. As a continuous implicit function Input, output three-dimensional floating-point coordinates The truncation distance value to the nearest object surface is denoted as the TSDF value; this truncation distance value is obtained by considering all objects that meet the criteria. The surface constructed from the points is defined as a three-dimensional continuous surface, providing a continuous geometric representation for subsequent numerical solutions; among which, In this context, x represents the X-axis coordinate value, and y represents the Y-axis coordinate value. This represents the Z-axis coordinate value.
[0079] Using the trained NeRF model of neural radiation field and a three-dimensional continuous surface, a ray-stepping solver is employed as the specific implementation method for iterative solving along the line-of-sight direction; for example... Figure 5 As shown, firstly, the optical center of the left camera... The origin is defined by the left camera of the stereo camera, where the subscript 'left' indicates the coordinates of the pixel being measured. A light beam is emitted from the left camera, traversing three-dimensional space, based on the coordinates of the pixel being measured. Together with the direction vector D, a three-dimensional spatial ray parameterized by parameter t is defined. :
[0080] ;
[0081] Select an initial point on the ray that is close to the camera as the current iteration point, denoted as . The subscript current indicates the position point in the current iteration step.
[0082] Current point The coordinates are input into the trained NeRF model of neural radiation field. In, among them, The subscript represents the continuous mapping function used to construct the neural network. This represents the weights contained in the neural network; NeRF model of neural radiation field. As a concrete manifestation of continuous implicit functions, it outputs the current point. The cutoff distance function value to the nearest object surface is denoted as s, i.e. .
[0083] According to the NeRF model of neural radiation fields The output truncation sign distance function value s, iteratively stepping along the ray direction D, the next iteration center point. The calculation formula is:
[0084] ;
[0085] In the formula, The next point is the center point for the next prediction iteration step, and the subscript next indicates the prediction position point for the next iteration step.
[0086] At the initial position In the diagram, subscript 0 represents the initial position number; NeRF model of neural radiation field. If the predicted truncation symbol distance function value s is large, a safe step size is determined based on the truncation symbol distance function value s, and the current coordinates are updated to the next position along the ray direction. The subscript 1 represents the first step position number; as the current position approaches the target surface, the NeRF model of neural radiation field... The step size is dynamically reduced and updated based on the decrease of the truncated symbol distance function value s, gradually approaching the target surface within the range of the three-dimensional continuous curved surface space; when the NeRF model of neural radiation field... When the truncated symbol distance function value s of the output converges to a preset threshold, that is, when the condition is met... When the convergence condition is met, determine whether the current iteration point touches the object surface, stop the iteration, and record the iteration point at this moment. This iteration point That is, the final iteration point of the line of sight ray and the three-dimensional continuous surface of the scene is identified, the Z-axis coordinate value of the final iteration point is extracted, and the Z-axis coordinate value of the intersection point is recorded as the sub-pixel precision depth value.
[0087] Traverse every pixel within the target region and utilize the trained NeRF neural radiation field model. Calculate the corresponding spatial geometric intersection points and extract the Z-axis coordinate values of each intersection point, which are recorded as the desired subpixel precision depth values. Collect all extracted subpixel precision depth values to construct a sample set composed of subpixel precision depth values. Through the sample set composed of subpixel precision depth values, a depth value list is compiled. The depth value list is then aggregated using a median voting mechanism. The median value selected by the aggregation decision is recorded as the final depth value. This final depth value is then output and recorded as the final long-distance measurement result.
[0088] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.
Claims
1. A long-distance ranging method based on binocular vision, characterized in that, This method utilizes an improved YOLOv11 model for long-distance ranging. The improvement adds an attention module, a coarse matching module, and a fine matching module to the original YOLOv11 model. The attention module performs horizontal and vertical pooling on the feature map to fuse and output image information, enhancing the improved YOLOv11 model's ability to accurately locate distant targets. The coarse matching module receives image information from the attention module, encodes the pixels in the feature map to generate encoding groups, and outputs discrete pixel pairs in integer pixel units, providing an initial basis for fine matching. The fine matching module iteratively trains the discrete pixel pairs obtained from the coarse matching module through multiple rounds, constructing the scene geometry as a continuous implicit function. By numerically analyzing this continuous implicit function, the discrete pixel pairs are transformed into continuous pixel regions, and depth values are extracted to achieve sub-pixel accuracy ranging. The specific implementation of the long-distance ranging method includes the following steps: Collect and preprocess pedestrian images from pre-detection to obtain corrected pedestrian images; The corrected pedestrian images are processed using a pre-trained improved YOLOv11 model as follows: The pedestrian image is input into the backbone network of the improved YOLOv11 model for feature extraction, and the original feature map is output. The original feature map is transmitted to the attention module, which performs horizontal and vertical pooling and weighting on the original feature map to generate an enhanced feature map, which is then transmitted to the detection head of the improved YOLOv11 model. The detection head outputs a series of target detection boxes based on the enhanced feature map, and uses the weighted box fusion algorithm in the attention module to fuse the series of target detection boxes to output the final two-dimensional bounding box. The coarse matching module receives the two-dimensional bounding box, and extracts the corresponding core region in the left and right views after stereo correction based on the two-dimensional bounding box. It then performs feature point matching and cost aggregation on the core region and outputs a series of discrete pixel pairs. The fine matching module constructs a local 3D bounding space containing the target object by combining a series of discrete pixel pairs obtained by the coarse matching module and the 2D bounding box obtained by the attention module. The truncated signed distance function is used as the geometric representation to construct the scene geometry within the local 3D bounding space as a continuous implicit function. Through real-time supervised training on the local 3D bounding space, a series of discrete pixel pairs are fitted and transformed into a continuous 3D surface. The spatial intersection of this continuous 3D surface and the line of sight is solved iteratively along the line of sight. The depth value of the spatial intersection is aggregated and decided to output the final ranging result.
2. The long-distance ranging method based on binocular vision according to claim 1, characterized in that, Preprocessing the acquired pedestrian images includes the following steps: Data Acquisition: A large number of multi-pose chessboard images were acquired using a binocular camera; Intrinsic parameter calibration: Using the chessboard image, single-target calibration is performed on the left and right cameras respectively, and the intrinsic parameters of the left and right cameras are calculated. Extrinsic parameter calibration: Based on the intrinsic parameters, the binocular camera is calibrated to obtain the rotation and translation matrix describing the relative positional relationship between the left and right cameras, i.e., the extrinsic parameters; Epipolar correction: Based on the intrinsic and extrinsic parameters, calculate the stereo correction transformation matrix so that the corrected left and right views satisfy the constraint that the epipolar lines are parallel and aligned. Stereo correction: Acquire pre-detected pedestrian images, use the acquired intrinsic parameters, extrinsic parameters and stereo correction transformation matrix to perform stereo correction on the input pedestrian images, record and output the corrected pedestrian images.
3. The long-distance ranging method based on binocular vision according to claim 1, characterized in that, The attention module is mainly composed of coordinate attention (CA) units responsible for filtering, and is equipped with a dynamic non-monotonic focusing (WIoU) loss function for training and optimization, and a weighted box fusion (WBF) algorithm for output fusion. The coarse matching module consists of a Census transformation method responsible for localization and a cost aggregation unit based on the principle of regional consistency. The fine matching module consists of a NeRF model of neural radiation field characterized by the truncated symbolic distance function TSDF and a ray stepping solver, wherein the NeRF model of neural radiation field is a specific implementation of a continuous implicit function and the ray stepping solver is a specific implementation of iterative solution.
4. The long-distance ranging method based on binocular vision according to claim 3, characterized in that, The attention module uses the coordinate attention (CA) unit to perform global average pooling in the horizontal and vertical directions on the corrected pedestrian image, generating feature maps with position awareness and orientation information. The dynamic non-monotonic focusing WIoU loss function records a large number of predicted boxes containing pedestrian target positions and categories based on the generated feature map with position awareness and orientation information during the model training phase. By calculating the confidence of the predicted boxes, the weight of simple samples is dynamically reduced, and a series of target detection boxes and their confidence scores are output. In the detection output stage, the weighted bounding box (WBF) algorithm clusters a series of target detection boxes according to their spatial location and intersection-union ratio (IU). For each cluster, the boundary coordinates of the target detection boxes are weighted and averaged according to their confidence scores. The target detection boxes are then fused, and the coordinates and confidence scores of the fused target detection boxes are calculated. All target detection boxes are then fused into the final two-dimensional bounding box.
5. The long-distance ranging method based on binocular vision according to claim 3, characterized in that, The coarse matching module receives the two-dimensional bounding box obtained by the attention module, segments the target and background in the corrected left and right views, extracts the core region, and uses the core region as input to the Census transform method for calculation and encoding; a core region mask matrix is defined as: ； Based on the aforementioned core region mask matrix, four center points and four support points within their neighborhoods are selected. Census transform binary encoding is performed on the center points and the four support points to generate a coding group containing five sets of codes. Coding group A and coding group B are generated for the left and right views, respectively. To calculate the initial matching cost, for each pixel to be matched, Census transform binary encoding is iterated multiple times within a preset disparity search range. In each iteration, for a preset disparity value, the coding values of coding group A and coding group B are compared bit by bit, and the number of all different bits is summed. This summation value is recorded as the Hamming distance value. The Hamming distance value of each coding group is recorded as the initial matching cost, and all initial matching costs are then constructed into an initial cost volume. The aforementioned core region mask matrix... The number is a random number representing the pixel value. The subscripts 1 to 21 indicate the position index numbers of the pixels involved in the calculation, arranged in order. The four support points are selected from the four pixels above, below, left, and right of the center point. The cost aggregation unit based on the regional consistency principle adopts a preset aggregation window. Based on the initial cost body, it accumulates and sums the initial matching costs of all pixels in the window under the same disparity, transforms the initial matching costs into aggregated matching costs, and thus constructs an aggregated cost body from all aggregated matching costs. Finally, iterating through the aggregation cost volume, for each pixel in the left and right views, the integer disparity that minimizes its aggregation matching cost is selected and denoted as the optimal matching disparity for that pixel. , where i represents the i-th pixel; calculate the minimum aggregate matching cost for all pixels, obtain the best matching disparity for each pixel, convert the disparity of each pixel into a depth value, and output a series of sparse discrete coarse matching point pairs, i.e., a series of discrete pixel pairs, where the disparity-depth conversion process is represented as: ； In the formula, represents the Z-axis coordinate of the pixel, b represents the baseline length of the stereo camera, f is the focal length of the stereo camera, and d represents the disparity of the pixel.
6. The long-distance ranging method based on binocular vision according to claim 3, characterized in that, The fine matching module processes a series of discrete pixel pairs obtained by the coarse matching module and the two-dimensional bounding box obtained by the attention module, including the following steps: For all input discrete pixel pairs Processing is carried out, among which The coordinates of the point are the X-axis coordinates. The coordinates are the Y-axis coordinates of the points, with subscript l indicating the left view and subscript r indicating the right view; extract the best matching disparity for each pair of points. Construct a disparity sample set; calculate the median of this disparity sample set, denoted as the central disparity. Where the subscript median indicates the median of the statistics; the central disparity Convert to scene center depth The subscript "center" indicates the location of the scene center; the depth of the scene center is... It will serve as the benchmark for constructing a local three-dimensional space; Based on center depth Extend the preset buffer distance Determine the near plane and far plane Wherein, the subscript near represents the near plane boundary of the three-dimensional space, and the subscript far represents the far plane boundary of the three-dimensional space; the Z-axis boundary of the three-dimensional space is defined; using the intrinsic parameters of the binocular camera, the four corner points of the two-dimensional bounding box are back-projected onto... and On these two depth planes, eight vertices in three-dimensional space are obtained; by calculating the maximum and minimum coordinates of the eight vertices on the X and Y axes, and combining them with the defined depth range, a minimum three-dimensional bounding space is constructed. The minimum three-dimensional bounding space is projected back onto the stereo-corrected left and right view images, respectively, and dense image patches containing complete visual information of the local scene are cropped out. Using these dense image patches as supervision information, and employing the truncated signed distance function (TSDF) as the internal geometric representation, a neural radiation field (NeRF) model is constructed to concretize the continuous implicit function. This NeRF model is then subjected to instantaneous iterative training within the minimum three-dimensional bounding space, and the resulting continuous implicit function is denoted as... The NeRF model of neural radiation field uses three-dimensional floating-point coordinates. As a continuous implicit function Input, output three-dimensional floating-point coordinates The truncation distance value to the nearest object surface is denoted as the TSDF value; this truncation distance value is obtained by considering all objects that meet the criteria. The surface constructed from the points is defined as a three-dimensional continuous surface, providing a continuous geometric representation for subsequent numerical solutions; among which, In this context, x represents the X-axis coordinate value, and y represents the Y-axis coordinate value. This represents the Z-axis coordinate value.
7. The long-distance ranging method based on binocular vision according to claim 3, characterized in that, The fine-matching module utilizes the trained NeRF model of neural radiation field and a three-dimensional continuous surface, employing a ray-stepping solver as the specific implementation for the iterative solution along the line-of-sight direction, with the optical center of the left camera as the reference point. The origin is defined as follows: where the subscript `left` indicates the left camera of the stereo camera; the coordinates of the two-dimensional pixel point in the image that requires depth query are denoted as... u represents the X-axis coordinate of a two-dimensional pixel, and v represents the Y-axis coordinate of a two-dimensional pixel; using the intrinsic parameters of a stereo camera, this two-dimensional pixel coordinate... Perform a back projection transformation and calculate from The direction vector D to the center of this two-dimensional pixel is the ray direction; with the left camera optical center... Together with the direction vector D, a three-dimensional spatial ray parameterized by parameter t is defined. : ； To find the 3D intersection point of the TSDF value convergence minimum threshold, points on the ray close to the camera are selected. Begin iterative stepping, where, The current iteration step center point, The current index represents the current position in the current iteration step, and the next index represents the predicted position in the next iteration step. The iterative calculation steps are as follows: 1) Query The TSDF value in the NeRF model of neural radiation field is obtained by denoting the truncated symbol distance as s: ； 2) Proceed a distance s along the ray direction D, and predict the center point of the next iteration step. The location is shown below: ； 3) Repeat steps 1) and 2) until s converges to the minimum threshold, and record the results. 3D intersection The subscript `final` indicates the final intersection point that satisfies the convergence condition; this 3D intersection point is recorded. The Z-axis coordinate value of the three-dimensional intersection point The Z-axis coordinate value is the desired sub-pixel precision depth value, and a sample set consisting of sub-pixel precision depth values is generated. 4) The sample set consisting of sub-pixel precision depth values resolved by the NeRF model of neural radiation field is compiled into a depth value list. The depth value list is aggregated and decided by a median voting mechanism. The median value selected by the aggregation decision is recorded as the final depth value. The final depth value is output and recorded as the final long-distance measurement result.