A method, device, and medium for target ranging based on binocular images

Abstract

Embodiments of the present application disclose a target ranging method and device based on binocular images and a medium, relating to the technical fields of target ranging, image matching and the like. The method comprises: taking a target frame of an object to be ranged in a reference image as a reference frame, and finding a first matching frame in a matching image by using a template matching method; dividing the reference frame into a plurality of first sub-regions, searching for second sub-regions most similar to the features of the first sub-regions, and excluding first sub-regions with inconsistent spatial distribution; determining a second matching frame of the reference frame according to the feature similarity of the remaining first sub-regions; selecting one of the first matching frame and the second matching frame with the largest intersection-over-union ratio with the second target frame as a final matching frame; determining the parallax of the object in the binocular image according to the coordinate offset of the final matching frame and the reference frame, and determining the radial distance of the object according to the parallax. The embodiments improve the accuracy and stability of binocular ranging.

Description

Technical Field

[0001] The embodiments of the present invention relate to the technical fields of target ranging and image matching, and in particular to a target ranging method, device and medium based on binocular images. Background Technology

[0002] Disparity map-based ranging methods primarily rely on binocular vision technology, using two cameras to acquire images and calculate disparity to determine target distance. Key technical aspects include camera calibration and stereo matching, the latter utilizing traditional algorithms such as SGBM (Semi-Global Block Matching) to calculate the disparity map. This method has wide applications in fields such as drones, autonomous driving, and robot navigation.

[0003] However, traditional algorithms such as SGBM have significant limitations when processing targets like drones. For example, isolated targets have little texture, making it difficult for the algorithm to find enough matching feature points, which can easily lead to disparity calculation errors; when the contrast between the target and the background is low, block matching is easily affected by background interference, resulting in blurred edges of the disparity map; and it has poor adaptability to changes in target attitude, such as tilting or rotating the drone, which can easily lead to matching failures and affect ranging accuracy. All these factors contribute to a decrease in the accuracy and stability of target ranging. Summary of the Invention

[0004] This invention provides a target ranging method, device, and medium based on binocular images to improve the accuracy and stability of target ranging.

[0005] In a first aspect, embodiments of the present invention provide a target ranging method based on binocular images, comprising:

[0006] S10. Obtain the stereo images to be processed as the reference image and the matching image, respectively;

[0007] S20. Using the first target box of the object to be measured in the reference image as the reference box, the template matching method is used to find the first matching box that matches the reference box in the matching image.

[0008] S30. Divide the reference box into multiple first sub-regions, and define a search region within the matching image that covers the second target box of the object in the matching image. Search within the search region for the second sub-region that is most similar to the features of each first sub-region.

[0009] S40. Compare the spatial distribution of each second sub-region in the matching image with the spatial distribution of each first sub-region in the reference image, and exclude the first sub-regions with inconsistent spatial distribution.

[0010] S50. Based on the feature similarity between each remaining first sub-region and its corresponding second sub-region, the coordinate offsets between each remaining first sub-region and its corresponding second sub-region are weighted and averaged; and based on the weighted average coordinate offsets, the second matching box of the reference box is determined.

[0011] S60. Select the one with the largest intersection-union ratio with the second target box from the first matching box and the second matching box, and use it as the final matching box;

[0012] S70. Based on the coordinate offset between the final matching box and the reference box, determine the disparity of the object in the binocular image, and determine the radial distance of the object based on the disparity.

[0013] In a second aspect, embodiments of the present invention provide an electronic device, the electronic device comprising:

[0014] One or more processors;

[0015] Memory, used to store one or more programs;

[0016] When the one or more programs are executed by the one or more processors, the one or more processors implement the target ranging method based on binocular images as described in any embodiment.

[0017] Thirdly, embodiments of the present invention also provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the target ranging method based on binocular images as described in any embodiment.

[0018] In summary, this invention provides a target ranging method based on binocular images. First, it coarsely locates the target region using cross-correlation template matching. Then, it employs particle segmentation to divide the reference bounding box into multiple sub-regions, and matches each sub-region sequentially within a sliding window of the matching image based on feature similarity. After matching, the spatial distribution order of the sub-regions verifies the result of eliminating geometric inconsistencies, and the final matching box is determined by the weighted summation of matching confidence scores, intersection-union ratio, and overlap ratio, achieving accurate disparity estimation and target ranging. The entire method, through a strategy of rapid initialization via template matching, local optimization via particle segmentation, and multi-feature cross-validation, balances real-time performance with anti-interference capabilities, and has the following technical advantages:

[0019] 1. High accuracy: By using the feature similarity of particle blocks, combined with particle order verification and dual index screening, it effectively overcomes changes in lighting and local occlusion, resulting in more accurate distance measurement;

[0020] 2. Strong anti-interference: It combines template matching initialization with particle block local optimization, taking into account both global search reliability and local matching sensitivity;

[0021] 3. High efficiency: The algorithm is lightweight and requires no training data or GPU (Graphics Processing Unit) support, enabling real-time processing with limited computing power, and has outstanding practical value in engineering.

[0022] In particular, due to significant signal interference during overall target bounding box matching, poor matching in any local area can lead to overall matching failure. Therefore, this embodiment introduces particle segmentation for fine-grained matching, making it easier to match between local image blocks and improving matching accuracy and stability. However, for objects like drones with multiple identical structures (e.g., the four wind turbines are a significant common structure in drones), particle segmentation matching will divide the data into multiple structurally similar sub-regions, leading to chaotic matching between these sub-regions. Therefore, this embodiment introduces spatial distribution verification, eliminating incorrect matches through the spatial geometric relationship between sub-regions (sequential verification), and determining the optimal result through weighted average of matching confidence and intersection-union ratio (IUU) overlap, thereby improving the accuracy and stability of binocular matching and target ranging. Attached Figure Description

[0023] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0024] Figure 1 This is a flowchart of a target ranging method based on binocular images provided in an embodiment of the present invention;

[0025] Figure 2 This is a flowchart of another target ranging method based on binocular images provided in an embodiment of the present invention;

[0026] Figure 3 This is a flowchart provided by an embodiment of the present invention for determining the final parallax based on the offset of a combination of multiple boxes;

[0027] Figure 4(a) is a comparison chart of the measured distance and the actual distance after target ranging is performed using the method of this embodiment;

[0028] Figure 4(b) is a comparison between the measured distance and the actual distance after target ranging using the SGBM algorithm;

[0029] Figure 4(c) is a comparison of the measured distance and the true distance after using the absolute values ​​of the coordinate offsets of the two original target boxes in the binocular image as disparity for target ranging.

[0030] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0031] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0032] In the description of this invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing the invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0033] In the description of this invention, it should also be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0034] Figure 1 This is a flowchart of a target ranging method based on binocular images provided by an embodiment of the present invention. The method first uses template matching to quickly locate the target region, then employs particle segmentation for precise matching, and fuses Sobel edge and RGB color features for weighted scoring matching. Finally, it undergoes multi-index verification and filtering to improve matching accuracy and anti-interference capability, thus meeting the real-time and accurate ranging requirements in complex scenarios such as UAVs. This method is executed by electronic devices, such as… Figure 1 As shown, the method specifically includes:

[0035] S110. Obtain the stereo images to be processed as the reference image and the matching image, respectively.

[0036] This embodiment first acquires binocular images at the same time, and uses one of the monocular images as the reference image and the other monocular image as the matching image for subsequent target box matching.

[0037] S120. Using the target bounding box of the object to be measured in the reference image as the reference bounding box, a matching bounding box matching the reference bounding box is found in the matching image using the template matching method.

[0038] In this embodiment, a target detection algorithm is used to obtain the bounding boxes of at least one target object in each of the two monocular images. Optionally, the bounding box of the same target object in the left image can be denoted as left_box, and the bounding box in the right image can be denoted as right_box. This step involves preliminary matching of the two bounding boxes belonging to the same object in the two monocular images.

[0039] In one specific implementation, the initial matching may include the following steps:

[0040] Step 1: Use the target bounding box of the object to be measured in the reference image as the reference bounding box, and define a search range in the matching image based on the reference bounding box and the maximum disparity.

[0041] Optionally, taking the left target image as the reference image as an example, the left-eye ROI (Region of Interest) is first extracted. From multiple bounding boxes in the left-eye image, the bounding box containing the object to be measured can be selected as the ROI_left, which is the reference bounding box.

[0042] Then, the right-eye search range is defined. Based on the horizontal coordinates of the reference frame and the maximum disparity (a known quantity), a search range roi_right_search is defined in the right-eye image. The horizontal start coordinate of this search range is right_search_start, and the horizontal end coordinate is right_search_end. The vertical coordinate range of this search range is consistent with the vertical coordinate range of the region of interest.

[0043] Step 2: Within the search range, perform template matching with the baseline bounding box using a normalized cross-correlation algorithm to obtain the optimal matching position. Optionally, within the search range roi_right_search, perform template matching with roi_left using a normalized cross-correlation algorithm (cv2.TM_CCOEFF_NORMED) to obtain the optimal matching position max_loc.

[0044] Step 3: Based on the optimal matching position, determine a matching box in the matching image that matches the reference box. Specifically, move the region in the matching image with the same coordinate as roi_left along the horizontal direction by max_loc to obtain the matching box in the matching image.

[0045] In this embodiment, the above preliminary matching is also referred to as template matching coarse positioning, and the resulting matching box is referred to as coarse positioning matching box.

[0046] S130. Divide the reference box into multiple sub-regions, and define a search region within the matching image that covers the target box of the object in the matching image. Within the search region, search for multiple other sub-regions that are most similar to the features of each sub-region.

[0047] This step involves dividing the reference bounding box into particle blocks and searching for image blocks in the matching image that match each block, preparing for subsequent secondary matching of the reference bounding box. For ease of distinction and description, this embodiment refers to the multiple sub-regions obtained by dividing the reference bounding box as the first sub-region, and the image blocks in the matching image that match each first sub-region as the second sub-region (i.e., the aforementioned "multiple other sub-regions").

[0048] In one specific embodiment, the above-mentioned particle block matching process may include the following steps:

[0049] Step 1: Divide the reference frame into multiple first sub-regions evenly along the same direction to complete particle initialization. Optionally, to balance factors such as computing power, local information loss, and matching accuracy, the reference frame can be evenly divided into 7 particles, i.e., 7 first sub-regions, along the horizontal direction.

[0050] Step 2: Using the bounding box of the object in the matching image as the center, define a search area within the matching image. This step determines the search center and search range in the matching image based on the object bounding box obtained by the object detection algorithm. The correspondence between the bounding boxes of the same object in the binocular images is known. For ease of distinction and description, this embodiment refers to the bounding box of the same object in the reference image obtained by the object detection algorithm in S120 as the first bounding box, and the bounding box in the matching image as the second bounding box. Optionally, still using the left target image as the reference image as an example, a dynamic search range, search_range, can be defined within the right target image, extending horizontally from the center of the right target box (right_box).

[0051] Step 3: For each first sub-region, perform operations S1 to S3 respectively:

[0052] S1. Within the search area, determine multiple candidate sub-regions with the same size as the current first sub-region, and the set of these candidate sub-regions fully covers the search area. Optionally, slide horizontally within the search range (search_range) and sequentially calculate multiple candidate offsets, each corresponding to a candidate sub-region in the matching image.

[0053] S2. Calculate the edge similarity and color similarity between each candidate sub-region and the current first sub-region, and determine the feature similarity based on the edge similarity and color similarity.

[0054] Optionally, image preprocessing can be performed first, calculating the Sobel edge feature maps of the left and right eye images respectively; and converting the left and right eye images into grayscale images respectively, separating the three color channels B (blue), G (green), and R (red).

[0055] Then, for each candidate sub-region: from the Sobel edge feature map of the reference image, the edge feature block corresponding to the current first sub-region is extracted; from the Sobel edge feature map of the matching image, the edge feature block corresponding to the current candidate sub-region is extracted; the difference between the two edge feature blocks is used to characterize the edge similarity between the current first sub-region and the current candidate sub-region. Simultaneously, from the multi-channel color map of the reference image, the color feature block corresponding to the current first sub-region is extracted; from the multi-channel color map of the matching image, the color feature block corresponding to the current candidate sub-region is extracted; the difference between the two color feature blocks is used to characterize the color similarity between the current first sub-region and the current candidate sub-region. The feature similarity between the current first sub-region and the current candidate sub-region is determined by weighted fusion of the two differences. For example, the sum of absolute errors of the two edge feature blocks and the sum of absolute errors of the two color feature blocks are calculated separately, and the two sums of absolute errors are weighted and fused with a weight of 60% (edge) + 40% (color) to obtain the total score. The lower the score, the higher the feature similarity.

[0056] S3. Select the candidate sub-region with the highest feature similarity as the second sub-region of the current first sub-region. Through S2, the feature similarity between each candidate sub-region and the current first sub-region can be obtained, and the candidate sub-region with the highest similarity is selected as the second sub-region.

[0057] Compared to the overall matching of the target box in S120, which involves more signal interference and can lead to overall matching failure due to poor matching in any local area, the particle block matching in this step focuses on local information, making it easier to match between local image blocks. Because the matching granularity after particle block matching is finer, this embodiment also refers to the matching in S130 as fine-grained particle block matching.

[0058] S140. Compare the spatial distribution of each second sub-region in the matching image with the spatial distribution of each first sub-region in the reference image, and exclude the first sub-regions with inconsistent spatial distribution.

[0059] This step performs sequential verification of the fine matching of particle blocks. When each first sub-region is distributed sequentially along the same direction (e.g., horizontal direction), each second sub-region in the matching image should also be distributed sequentially along the same direction, and the order of each second sub-region should be consistent with the order of each first sub-region. If this sequentiality (geometric constraint) is not met, the first and second sub-regions that do not meet the sequentiality are excluded.

[0060] For example, if the first sub-regions are numbered sequentially as 1, 2, 3, 4, 5, 6, 7, and the corresponding second sub-regions are numbered sequentially as 1, 2, 5, 4, 3, 6, 7 (sub-regions numbered 3 and 5 do not satisfy the order constraint), then the first and second sub-regions numbered 3 and 5 should be excluded.

[0061] S150. Based on the feature similarity between the remaining first sub-regions and the corresponding second sub-regions, perform a weighted average of the coordinate offsets between the remaining first sub-regions and the corresponding second sub-regions; and determine another matching box of the reference box based on the weighted average coordinate offset.

[0062] This step performs a secondary matching of the baseline box based on the matching results of particle segmentation. In this embodiment, the "other matching box" obtained by the secondary matching is called the refined matching box.

[0063] In one specific implementation, for all remaining first sub-regions, they are first sorted in descending order based on the matching confidence (i.e., feature similarity) between each first sub-region and its corresponding second sub-region. Each first sub-region is then assigned a corresponding weight according to the sorting result, with higher confidence levels resulting in a larger weight. All weights are then normalized to ensure that the sum of the particle weights for each first sub-region after normalization is 1. Next, the coordinate offset (two-dimensional image coordinate offset) between each remaining first sub-region and its corresponding second sub-region is calculated. These offsets are multiplied by the normalized weight of the particle, and the "offset-weight product" results for all first sub-regions are summed to obtain the weighted fused coordinate offset. Finally, the coordinates of the reference box are offset based on this weighted fused coordinate offset to obtain another matching box in the matching image.

[0064] For example, taking the left eye image as the reference image, the offset after weighted fusion can be denoted as refined_right_shift_candidate. Using this offset, the coordinates of the reference box are offset to obtain the horizontal coordinates refined_right_x1 of the refined matching box in the right eye image. Keeping the width and height of the right box consistent with the reference box in the left eye image, a refined matching box refined_right_box2 is formed.

[0065] For ease of distinction and description, this embodiment also refers to the coarse positioning matching box obtained in S120 as the first matching box, and the refined matching box obtained in this step as the second matching box.

[0066] S160. Select the one with the largest intersection-union ratio with the second target box from the first matching box and the second matching box, and use it as the final matching box.

[0067] This step merges and outputs the two matching boxes. Taking the left eye image as the reference image, the refined candidate box refined_right_box2 is compared with the coarse matching box coarse_match_box, and the one with the higher intersection-union ratio (IUU) with the original right_box is selected as the final matching box refined_right_box3.

[0068] Furthermore, after comparison and fusion, anomaly verification and boundary protection can be performed. This involves checking whether the absolute value of the coordinate offset between the final matching box and the second target box exceeds a set threshold, and whether the final matching box exceeds the boundary of the matching image. If it exceeds the set threshold or exceeds the boundary of the matching image, the second target box is still considered the final matching box.

[0069] For example, still using the left eye image as the reference image, we check the offset between the center of the final matching box refined_right_box3 and the center of the original right_box. If the offset is too large (e.g., more than 7 pixels), the refinement is deemed a failure, and we revert to the original right_box to ensure the robustness of the algorithm. At the same time, we ensure that the output box does not exceed the image boundary. Finally, we output the final matching box refined_right_box.

[0070] S170. Based on the coordinate offset between the final matching box and the reference box, determine the disparity of the object in the binocular image, and determine the radial distance of the object based on the disparity.

[0071] The offset of the center coordinates of the final matching box from the reference box represents the disparity of the object in the binocular image. The distance from the object to the binocular vision system can be calculated based on this disparity.

[0072] In one specific implementation, to improve the stability of the ranging results, the reference image and the matching image in S110 to S160 can be swapped to perform a new round of target box matching, and the disparity information can be updated based on the results of the two rounds of matching. For example... Figure 2As shown, the left and right images in the binocular images are first used as the reference image and the matching image, respectively. Operations S110 to S160 are performed to obtain the final matching box of the target box in the left image in the right image. Similarly, the right and left images in the binocular images are used as the reference image and the matching image, respectively, and operations S110 to S160 are performed to obtain the final matching box of the target box in the right image in the left image. It should be noted that in this embodiment, the first target box and the second target box correspond to the reference image and the matching image, respectively. When the reference image and the matching image are swapped, the specific references of the first target box and the second target box in the algorithm are also swapped.

[0073] Then, the target bounding boxes of the object in the left and right eye images are paired with the final matching boxes in the left and right eye images respectively, forming four box combinations. The absolute values ​​of the coordinate offsets of the two boxes in each combination are calculated, resulting in four absolute coordinate offset values. Finally, the absolute values ​​of the coordinate offsets corresponding to the two combinations from the binocular images are used as candidate disparities (i.e., candidate values ​​of disparity). The confidence of the two candidate disparities is determined based on the absolute values ​​of the coordinate offsets corresponding to the two combinations from the monocular image, and the final disparity of the object in the binocular images is determined based on the determination results.

[0074] Figure 3 This is a schematic diagram illustrating how the final disparity is determined based on the absolute values ​​of four offsets, as provided in an embodiment of the present invention. Figure 3 As shown, firstly, using the left eye image as the reference image and the target box left_box in the left eye image as the reference box, the final matching box refined_right_box corresponding to left_box is determined in the right eye image; similarly, using the right eye image as the reference image and the target box right_box in the right eye image as the reference box, the final matching box refined_left_box corresponding to right_box is determined in the left eye image.

[0075] Then, the target bounding box `left_box` and the final matching bounding box `refined_left_box` are combined into a single bounding box. The x-coordinate (horizontal coordinate) of the center point of `left_box`, `left_box_center_x`, is taken, and the absolute value of the difference between the x-coordinate of the center point of `refined_left_box` and the x-coordinate of the center point of `refined_left_box` is calculated as `left_refinedL_x_abs`. Since both the bounding box `left_box` and `refined_left_box` come from the left-eye image, `left_refinedL_x_abs` represents the absolute value of the coordinate offset from the monocular image.

[0076] Similarly, the target bounding box `right_box` and the final matching bounding box `refined_right_box` are combined into a single bounding box. The x-coordinate of the center point of `right_box` (`right_box_center_x`) and the x-coordinate of the center point of `refined_right_box` are then compared, and the absolute value of the difference between them (`right_refinedR_x_abs`) is calculated. Since both the bounding box combination `right_box` and `refined_right_box` originate from the right-eye image, `right_refinedR_x_abs` also represents the absolute value of the coordinate offset from the monocular image.

[0077] The target bounding box `left_box` and the final matching bounding box `refined_right_box` are combined into a single bounding box. The x-coordinate of the center point of `left_box` (`left_box_center_x`) and the x-coordinate of the center point of `refined_right_box` are taken, and the absolute value of the difference between them is calculated as `lr_box_x_center_diff_abs`. Since `left_box` and `refined_right_box` in this bounding box are derived from the left and right visual images, respectively, `lr_box_x_center_diff_abs` represents the absolute value of the coordinate offset from the binocular images, and is therefore a candidate disparity.

[0078] Similarly, the target bounding box (right_box) and the final matching bounding box (refined_left_box) are combined into a single bounding box. The x-coordinate of the center point of the right_box (right_box_center_x) is taken, and the x-coordinate of the center point of the refined_left_box is compared with that of the right_box, and the absolute value of the difference between them (rl_box_x_center_diff_abs) is calculated. Since the right_box and refined_left_box in this bounding box are derived from the right and left visual images, respectively, rl_box_x_center_diff_abs also belongs to the absolute value of coordinate offset from the binocular images, and is another candidate disparity.

[0079] After obtaining the absolute values ​​of the four coordinate offsets mentioned above, the two absolute values ​​of coordinate offsets from the monocular image are used as the confidence scores of the two candidate disparities. Specifically, the absolute value of the coordinate offset from the left image, `left_refinedL_x_abs`, is used as the confidence score of the candidate disparity `rl_box_x_center_diff_abs` calculated based on the final matching box `refined_left_box` in the left image. The smaller the coordinate offset `left_refinedL_x_abs` from the left image, the more reasonable the final matching box `refined_left_box` in the left image is, and the higher the confidence score of the candidate disparity `rl_box_x_center_diff_abs` calculated from `refined_left_box`. Similarly, the absolute value of the coordinate offset from the right image, `right_refinedR_x_abs`, is used as the confidence score of the candidate disparity `lr_box_x_center_diff_abs` calculated based on the final matching box `refined_right_box` in the right image. The smaller the coordinate offset right_refinedR_x_abs from the right eye image, the more reasonable the final matching box refined_right_box in the right eye image is, and the higher the confidence of the candidate disparity lr_box_x_center_diff_abs calculated from refined_right_box.

[0080] Based on the aforementioned representative relationship, and in conjunction with Figure 3 When determining disparity, first check whether left_refinedL_x_abs<5 and right_refinedR_x_abs<5, that is, check whether the confidence of the two candidate disparities is high enough.

[0081] If both candidate disparities satisfy (left_refinedL_x_abs < 5 and right_refinedR_x_abs < 5), then both have high confidence. Next, calculate the absolute value of the difference between the two candidate disparities, bi_disparity_diff_abs. If this absolute value < 3, it confirms that the difference between the two candidate disparities is within a reasonable range and has high confidence; therefore, the average of the two candidate disparities is taken as the final disparity. If this absolute value ≥ 3, then compare the magnitudes of left_refinedL_x_abs and right_refinedR_x_abs, and select the candidate disparity with the higher confidence as the final disparity.

[0082] If not (the condition "left_refinedL_x_abs < 5 and right_refinedR_x_abs < 5" is not met), then continue to check if left_refinedL_x_abs < 5 or right_refinedR_x_abs < 5, i.e., check if either of the two candidate boxes has a sufficiently high confidence level. If yes (left_refinedL_x_abs < 5 or right_refinedR_x_abs < 5), then continue to compare the values ​​of left_refinedL_x_abs and right_refinedR_x_abs, and use the candidate disparity with the higher confidence level as the final disparity. If no (left_refinedL_x_abs ≥ 5 and right_refinedR_x_abs ≥ 5), then backtrack to the two target boxes, and use the coordinate offset of the two target boxes, box_x_center_disparity, as the final disparity.

[0083] It should be noted that the thresholds 5 and 3 in the above judgment represent 5 pixels and 3 pixels respectively, and the values ​​of each threshold can be set as needed. If the three pixel thresholds of 7 pixels, 5 pixels, and 3 pixels mentioned above are denoted as the first threshold, the second threshold, and the third threshold respectively, then it is preferable to satisfy the condition that the first threshold > the second threshold > the third threshold, for example, 7 > 5 > 3.

[0084] After obtaining the final disparity, the 3D coordinates of the target object (3D coordinates in the camera coordinate system) are calculated to obtain the target's distance information. Specifically, given the binocular disparity d (the horizontal displacement difference of the object on the image plane), the binocular camera intrinsic parameters, and the baseline between the cameras (the horizontal distance between the binocular cameras), the target distance can be calculated through the following steps:

[0085] First, the depth of the object is calculated based on parallax. Parallax is the horizontal difference between corresponding points in two images, reflecting the distance between the object and the camera. The relationship between parallax and object depth can be described by the following formula:

[0086]

[0087] Where Z represents the object depth, i.e., the Z coordinate in the camera coordinate system; f represents the camera focal length, usually provided by intrinsic parameters; B represents the baseline of the stereo camera, i.e., the horizontal distance between the two cameras; and d represents the parallax value, i.e., the difference in horizontal displacement of the same object in two images.

[0088] Then, calculate the object's 3D coordinates in the camera coordinate system. For each pixel, assuming its 2D image coordinates are (x, y), its 3D coordinates in the camera coordinate system can be calculated using the following formula:

[0089]

[0090]

[0091] Where X and Y represent the coordinates of each pixel in the object in the camera coordinate system, corresponding to the width and height of the object, respectively. It is the position of the camera principal point (i.e., the intersection of the optical axis and the image plane) in the image.

[0092] Finally, calculate the radial distance. The radial distance is the straight-line distance between the object and the origin of the camera coordinate system, and can be calculated using 3D coordinates.

[0093]

[0094] Where R is the radial distance between the object and the origin of the camera coordinate system, which is the final ranging result in this embodiment.

[0095] Figures 4(a) to 4(c) Figure 4(a) compares the results of target ranging using the method of this embodiment with two other methods. All three figures correspond to the same binocular image sequence. Figure 4(a) compares the measured distance with the true distance after frame-by-frame target ranging using the method of this embodiment; Figure 4(b) compares the measured distance with the true distance after frame-by-frame target ranging using the SGBM algorithm; Figure 4(c) compares the measured distance with the true distance after frame-by-frame target ranging using the absolute values ​​of the coordinate offsets of the two original target boxes in the binocular image as disparity. It can be seen that the method of this embodiment significantly improves ranging accuracy and stability.

[0096] In summary, this embodiment provides a target ranging method based on binocular images. First, it coarsely locates the target region using normalized cross-correlation template matching. Then, it employs particle segmentation to divide the reference bounding box into multiple sub-regions. Each sub-region is matched sequentially within a sliding window of the matching image using a weighted score of Sobel edge features (e.g., 60% weight) and RGB three-channel color features (e.g., 40% weight). After matching, the spatial distribution order of the sub-regions verifies the results of eliminating geometric inconsistencies, and the optimal offset is determined by the weighted summation of matching confidence scores, intersection-union ratio, and overlap ratio. Finally, dynamic boundary protection outputs a high-precision final matching box, achieving accurate disparity estimation and target ranging. The entire method, through a strategy of rapid initialization via template matching, local optimization via particle segmentation, and multi-feature cross-validation, balances real-time performance and anti-interference capabilities, offering the following technical advantages:

[0097] 1. High accuracy: By combining Sobel edge and RGB color weighted fusion with particle order verification and dual index screening, it effectively overcomes changes in lighting and local occlusion, resulting in more accurate distance measurement;

[0098] 2. Strong anti-interference: It combines template matching initialization with particle block local optimization to balance global search reliability and local matching sensitivity; it comprehensively determines the binocular time distance by combining the original binocular target box and the final binocular matching box, further improving the ranging stability.

[0099] 3. High Efficiency: The algorithm is lightweight, requiring no training data or GPU support, and achieves real-time processing with limited computing power, demonstrating significant practical engineering value. Compared to the method in this application, traditional globally optimized stereo matching algorithms construct cost cubes through mutual information or Census transformation and perform cost aggregation to obtain dense, high-precision disparity maps, but with a large computational load; while deep learning methods use end-to-end deep neural networks to directly regress disparity maps through feature extraction, cost construction, and 3D convolutional regularization, requiring a large amount of data for training and GPU computing power.

[0100] Specifically, addressing the difficulty of ranging for symmetrical target objects like drones with multiple identical structures, this implementation proposes several targeted improvements: First, due to significant signal interference during overall target bounding box matching, poor matching in any local area can lead to overall matching failure. Therefore, this embodiment introduces particle segmentation for fine-grained matching, making it easier to match between local image blocks and improving matching accuracy and stability. However, for objects like drones with multiple identical structures (e.g., the four wind turbines are a significant identical structure in a drone), particle segmentation matching can divide the target into multiple structurally similar sub-regions, leading to chaotic matching between these sub-regions. Therefore, this embodiment introduces spatial distribution verification, eliminating incorrect matches through the spatial geometric relationship (sequential verification) between sub-regions, and determining the optimal result through a weighted average of matching confidence and the intersection-union ratio (IUGR) overlap. Furthermore, this embodiment also employs multi-feature weighted scoring and fusion, utilizing Sobel edge features (60% weight) and RGB color features (40% weight) to improve matching accuracy; a dynamic boundary protection mechanism is implemented to prevent bounding boxes from exceeding limits when outputting the final matching box; and the binocular time distance is comprehensively determined by pairwise combinations of the original binocular target boxes and the final binocular matching boxes. These measures are specifically designed to combat interference in symmetrical target objects like UAVs with multiple identical structures, aiming to improve the accuracy and stability of binocular matching and target ranging.

[0101] It should be noted that all data involved in this application are information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation portals are provided for users to choose to authorize or refuse.

[0102] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention, such as... Figure 5 As shown, the device includes a processor 60, a memory 61, an input device 62, and an output device 63; the number of processors 60 in the device can be one or more. Figure 5 Taking a processor 60 as an example; the processor 60, memory 61, input device 62, and output device 63 in the device can be connected via a bus or other means. Figure 5 Taking the example of a connection between China and Israel via a bus.

[0103] The memory 61, as a computer-readable storage medium, can be used to store software programs, computer-executable programs, and modules, such as the program instructions / modules corresponding to the target ranging method based on binocular images in this embodiment of the invention. The processor 60 executes various functional applications and data processing of the device by running the software programs, instructions, and modules stored in the memory 61, thereby implementing the aforementioned target ranging method based on binocular images.

[0104] The memory 61 may primarily include a program storage area and a data storage area. The program storage area may store the operating system and at least one application program required for a given function; the data storage area may store data created based on terminal usage. Furthermore, the memory 61 may include high-speed random access memory and non-volatile memory, such as at least one disk storage device, flash memory, or other non-volatile solid-state storage device. In some instances, the memory 61 may further include memory remotely located relative to the processor 60, which can be connected to the device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0105] Input device 62 can be used to receive input digital or character information, and to generate key signal inputs related to user settings and function control of the device. Output device 63 may include display devices such as a display screen.

[0106] This invention also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the target ranging method based on binocular images of any embodiment.

[0107] The computer storage medium of this invention can be any combination of one or more computer-readable media. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0108] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0109] Program code contained on a computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0110] Computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof. Programming languages ​​include object-oriented programming languages—such as Java, Smalltalk, and C++—as well as conventional procedural programming languages—such as C or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0111] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the technical solutions of the embodiments of the present invention.

Claims

1. A target ranging method based on binocular images, characterized in that, include: S10. Obtain the stereo images to be processed as the reference image and the matching image, respectively; S20. Using the first target box of the object to be measured in the reference image as the reference box, the template matching method is used to find the first matching box that matches the reference box in the matching image. S30. Divide the reference box into multiple first sub-regions, and define a search region within the matching image that covers the second target box of the object in the matching image. Search within the search region for the second sub-region that is most similar to the features of each first sub-region. S40. Compare the spatial distribution of each second sub-region in the matching image with the spatial distribution of each first sub-region in the reference image, and exclude the first sub-regions with inconsistent spatial distribution. S50. Based on the feature similarity between each remaining first sub-region and its corresponding second sub-region, the coordinate offsets between each remaining first sub-region and its corresponding second sub-region are weighted and averaged; and based on the weighted average coordinate offsets, the second matching box of the reference box is determined. S60. Select the one with the largest intersection-union ratio with the second target box from the first matching box and the second matching box, and use it as the final matching box; S70. Based on the coordinate offset between the final matching box and the reference box, determine the disparity of the object in the binocular image, and determine the radial distance of the object based on the disparity.

2. The method according to claim 1, characterized in that, The object is a collection of objects with the same structure; The object includes a drone, and multiple identical structures of the drone include multiple wind turbines.

3. The method according to claim 1, characterized in that, The step of using template matching to find a first matching box in the matching image that matches the reference box includes: Based on the reference frame and the maximum disparity, the search range is defined in the matching image; Within the search range, template matching is performed between the normalized cross-correlation algorithm and the baseline box to obtain the optimal matching position; Based on the optimal matching position, a first matching box that matches the reference box in the matching image is determined.

4. The method according to claim 1, characterized in that, Dividing the reference frame into multiple first sub-regions includes: uniformly dividing the reference frame into multiple first sub-regions along the same direction; Accordingly, S40 includes: comparing the order of each second sub-region in the matching image along the same direction with the order of each first sub-region in the reference image along the same direction, and excluding the first sub-regions with inconsistent order.

5. The method according to claim 1, characterized in that, Within the matching image, a search region is defined that covers the second target bounding box of the object in the matching image. Within the search region, a second sub-region that is most similar to the features of each first sub-region is searched, including: A search area is defined within the matching image, centered on the second target box of the object in the matching image. For each first sub-region: within the search area, determine multiple candidate sub-regions that have the same size as the current first sub-region and fully cover the search area. Calculate the edge similarity and color similarity between each candidate sub-region and the current first sub-region, and determine the feature similarity based on the edge similarity and color similarity. Select the candidate sub-region with the highest feature similarity as the second sub-region.

6. The method according to claim 5, characterized in that, The step of calculating the edge similarity and color similarity between each candidate sub-region and the current first sub-region, and determining the feature similarity based on the edge similarity and color similarity, includes: From the Sobel edge feature map of the reference image, extract the edge feature block corresponding to the current first sub-region; from the Sobel edge feature map of the matching image, extract the edge feature block corresponding to the current candidate sub-region; use the difference between the two edge feature blocks to characterize the edge similarity between the current first sub-region and the current candidate sub-region. From the multi-channel color map of the reference image, extract the color feature block corresponding to the current first sub-region; from the multi-channel color map of the matching image, extract the color feature block corresponding to the current candidate sub-region; use the difference between the two color feature blocks to characterize the color similarity between the current first sub-region and the current candidate sub-region. The two differences are weighted and fused to determine the feature similarity between the current first sub-region and the current candidate sub-region.

7. The method according to claim 1, characterized in that, Following the S60, it also includes: Check whether the absolute value of the coordinate offset between the final matching box and the second target box exceeds a set threshold, and whether the final matching box exceeds the boundary of the matching image; If the set threshold is exceeded, or the boundary of the matching image is exceeded, the second target box will still be used as the final matching box.

8. The method according to claim 1, characterized in that, Before S70, the method further includes: using the left and right eye images in the binocular images as the reference image and the matching image respectively, performing operations S10 to S60 to obtain the final matching box in the right eye image; using the right and left eye images in the binocular images as the reference image and the matching image respectively, performing operations S10 to S60 to obtain the final matching box in the left eye image. Accordingly, determining the disparity of the object in the binocular image based on the coordinate offset between the final matching box and the reference box includes: pairing the target box of the object in the left and right images with the final matching boxes in the left and right images respectively, and calculating the absolute value of the coordinate offset for each pair; using the absolute values ​​of the coordinate offsets corresponding to the two pairs from the binocular images as candidate disparities, judging the confidence of the two candidate disparities based on the absolute values ​​of the coordinate offsets corresponding to the two pairs from the monocular images, and determining the final disparity of the object in the binocular image based on the judgment result.

9. An electronic device, characterized in that, include: One or more processors; Memory, used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the target ranging method based on binocular images as described in any one of claims 1-8.

10. A computer-readable storage medium, characterized in that, It stores a computer program that, when executed by a processor, implements the target ranging method based on binocular images as described in any one of claims 1-8.

Images