A scene feature-based parameter adaptive matching binocular distance measuring method and system

Abstract

The application discloses a kind of parameter adaptive matching binocular ranging method and system based on scene feature, it relates to image matching technical field.The application realizes the comprehensive promotion of ranging accuracy and robustness by introducing scene characteristic evaluation and parameter adaptive mechanism.Under the driving of scene feature, the key parameters of BM algorithm are dynamically adjusted, which avoids the problem of easy mismatching of traditional fixed parameters in different scenes, thereby ensuring stable ranging effect in light change, low texture or dynamic environment, and getting rid of the dependence on manual parameter adjustment, significantly reducing engineering deployment and maintenance cost.Combining YOLO model for target detection, the class, position and depth map information are fused to realize automatic identification and ranging of specific target.

Description

Technical Field

[0001] This invention relates to the field of image matching technology, and in particular to a parameter adaptive matching binocular ranging method and system based on scene features. Background Technology

[0002] In the field of 3D environment perception, pure vision-based binocular ranging technology based on natural optical images offers advantages such as no need for active light sources and low cost. The prerequisite for binocular ranging is stereo matching of two images acquired simultaneously by the left and right eyes. Block matching (BM) is a commonly used method in this process, achieving fast computation through local matching, making it suitable for resource-constrained embedded devices or real-time processing scenarios. However, the matching performance of the BM algorithm is directly related to the rationality of parameter settings. The richness of image texture, the depth range of objects, and the degree of dynamic change over time all affect its matching performance. Manual parameter tuning based on scene changes is usually required, which limits the application of this algorithm in practical tasks. In binocular ranging, the parameters of the BM algorithm are not dynamically adjusted according to scene characteristics; instead, fixed parameter settings are relied upon based on human experience, lacking real-time adaptive capabilities. Therefore, changes in scene texture complexity, depth range, or dynamic changes can easily lead to a decline in matching performance. Meanwhile, this method does not introduce a target detection module, and the ranging results can only provide overall depth information, which cannot be associated with specific targets. Therefore, it cannot achieve automatic detection and ranging of specific targets, and its application flexibility and practicality are limited. Summary of the Invention

[0003] The purpose of this invention is to provide a parameter adaptive matching binocular ranging method and system based on scene features to improve the above-mentioned technical problems.

[0004] To achieve the above-mentioned objectives, the embodiments of the present invention provide the following technical solutions:

[0005] A scene feature-based parameter adaptive matching binocular ranging method, comprising:

[0006] Perform geometric calibration on the stereo camera, obtain the camera intrinsic and extrinsic parameter matrices, and establish the mapping relationship between pixel coordinates and actual spatial coordinates;

[0007] The left and right views of the current scene are acquired in real time using a geometrically calibrated binocular camera.

[0008] Perform stereo correction on the left and right views, and generate corresponding grayscale images through image conversion;

[0009] The scene characteristics of the current scene are evaluated based on any grayscale image, and the key parameters of the BM algorithm are dynamically adjusted. The key parameters include matching window size, texture threshold, uniqueness ratio, speckle range, disparity range, and preprocessing cutoff value. The scene characteristics include texture complexity, scene depth difference level, and scene motion degree.

[0010] The modified BM algorithm is used to perform stereo matching on two grayscale images. Using either grayscale image as a reference, target detection information is generated by combining it with the YOLO model, and the ranging result is output. The ranging result includes the category, position and distance information of the detected target.

[0011] In traditional binocular ranging based on the BM algorithm, parameter settings are fixed; however, fixed parameters directly impact matching performance. When scene texture complexity is low, depth range varies greatly, or significant motion exists, fixed parameters often lead to mismatches and inaccurate depth estimation, thus affecting ranging accuracy. Furthermore, existing parameter tuning methods typically require frequent manual adjustments, which are time-consuming and fail to meet real-time requirements, severely limiting their application in dynamic environments and embedded devices. Therefore, this invention introduces scene characteristics and adjusts key parameters of the BM algorithm in real time. By real-time evaluation of texture complexity, depth difference levels, and scene motion in the acquired images, matching errors caused by lighting changes and object motion are reduced, generating key parameters. This avoids manual parameter tuning, ensuring stable ranging performance even under varying lighting or dynamic interference environments, providing stronger adaptability and ensuring good matching accuracy and robustness of BM stereo matching in different environments.

[0012] Furthermore, traditional BM ranging only outputs a depth map and cannot be correlated with specific targets, thus limiting its applicability in applications requiring "detection + ranging" (such as autonomous driving, security monitoring, and robot grasping). Therefore, this invention, based on the generated depth map, combines target category and location information detected by the YOLO model to obtain the actual distance to the target. This not only provides overall depth information but also enables automatic identification and ranging of specific targets, enhancing the system's practicality in intelligent driving, monitoring, and robotics.

[0013] A scene-feature-based parameter adaptive matching binocular ranging system includes:

[0014] The pixel-space mapping relationship construction module is used to perform geometric calibration of the stereo camera, obtain the camera intrinsic and extrinsic parameter matrices, and establish the mapping relationship between pixel coordinates and actual spatial coordinates.

[0015] The left and right view real-time acquisition module is used to synchronously acquire the left and right views of the current scene in real time through a geometrically calibrated binocular camera;

[0016] The stereo correction module is used to perform stereo correction on the left and right views and generate corresponding grayscale images through image conversion.

[0017] The scene characteristic calculation module is used to evaluate the scene characteristics of the current scene based on any grayscale image;

[0018] The BM algorithm adaptive adjustment module is used to dynamically adjust the key parameters of the BM algorithm based on scene characteristics;

[0019] The stereo matching module is used to perform stereo matching on two grayscale images using the adjusted BM algorithm to generate a depth map;

[0020] The target detection information generation module is used to generate target detection information by taking any grayscale image as a reference and combining it with the YOLO model.

[0021] The ranging result calculation module is used to calculate the ranging result based on the depth map and target detection information.

[0022] This system achieves fully automated processing from camera calibration, image acquisition, and scene characteristic analysis to adaptive stereo matching and target detection fusion through modular design, avoiding the problems of multi-step manual intervention and independent tool combinations required in traditional methods. The functional modules are interconnected and have collaborative optimization capabilities, enabling the system to automatically adjust parameters and generate stable depth maps in different scenarios. Simultaneously, it combines target detection information to complete target-level ranging, exhibiting high integration, real-time performance, and scalability, significantly enhancing the intelligence and practical value of binocular ranging systems in complex application environments. Attached Figure Description

[0023] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 This is a flowchart of the method in Embodiment 1 of the present invention;

[0025] Figure 2 This is a diagram showing the relative positions of the checkerboard grid and the binocular camera during the binocular camera calibration in Embodiment 1 of the present invention.

[0026] Figure 3 This is a flowchart of the ranging results in Embodiment 1 of the present invention;

[0027] Figure 4 This is a system structure diagram of Embodiment 1 of the present invention;

[0028] Figure 5 This is the depth map in Embodiment 2 of the present invention;

[0029] Figure 6 This is a diagram showing the correspondence between the depth map and the acquired image in Embodiment 2 of the present invention;

[0030] Figure 7 This is a schematic diagram of the distance measurement results in Embodiment 2 of the present invention. Detailed Implementation

[0031] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0032] Example 1:

[0033] Please see Figure 1 This embodiment provides a parameter adaptive matching binocular ranging method based on scene features, including:

[0034] S1. Perform geometric calibration on the binocular camera, obtain the camera intrinsic and extrinsic parameter matrices, and establish the mapping relationship between pixel coordinates and actual spatial coordinates;

[0035] Specifically, such as Figure 2 As shown, establishing the mapping relationship between pixel coordinates and actual spatial coordinates includes:

[0036] S1-1. Acquire calibration images (chessboard pattern), perform single-target intrinsic parameter calibration of the stereo camera using Zhang Zhengyou calibration method or OpenCV calibration tool, and obtain the corresponding intrinsic parameter matrix;

[0037] S1-2. Match the corner coordinates of the stereo camera on the calibration image, calculate the rotation matrix and translation vector of the stereo camera, and construct the extrinsic parameter matrix of the stereo camera.

[0038] S1-3. Based on two intrinsic parameter matrices and two extrinsic parameter matrices, disparity and spatial depth are calculated by calibrating the pixel coordinates on the image, and a mapping relationship between pixel coordinates and actual spatial coordinates is established.

[0039] S2. Real-time synchronous acquisition of the left and right views of the current scene using a geometrically calibrated binocular camera;

[0040] S3. Perform stereo correction on the left and right views, and generate corresponding grayscale images through image conversion;

[0041] Specifically, geometric correction is performed on the left and right views acquired by the binocular camera to ensure that the corresponding epipolar lines of the two images are strictly parallel, guaranteeing that points of the same scene lie on the same horizontal row in both images, thus simplifying the parallax search process. During the correction process, perspective transformation is performed on the images using the camera's intrinsic and extrinsic parameter matrices and distortion parameters to eliminate lens distortion and differences in shooting angles. Subsequently, the corrected color image or the original image is converted into a grayscale image, and brightness information is extracted to reduce data dimensionality and noise interference, providing stable and efficient input for the subsequent stereo matching of the BM algorithm.

[0042] Geometric correction ensures epipolar alignment between the left and right images, reducing parallax search complexity and improving matching accuracy. Grayscale processing reduces the impact of lighting and color differences, enhancing the stability and robustness of the BM algorithm under different texture and lighting conditions, while also improving computational efficiency.

[0043] S4. Evaluate the scene characteristics of the current scene based on any grayscale image, and dynamically adjust the key parameters of the BM algorithm; the key parameters include matching window size, texture threshold, uniqueness ratio, speckle range, disparity range, and preprocessing truncation value; the scene characteristics include texture complexity, scene depth difference level, and scene motion degree.

[0044] Table 1

[0045]

[0046] In stereo matching, different scene characteristics directly affect the difficulty and accuracy of matching. Specifically, texture complexity reflects the richness of local features in an image. The richer the texture, the easier and less prone to confusion in matching. Therefore, the texture threshold can be increased and the uniqueness ratio relaxed, while using a smaller matching window to retain more details. Conversely, when the texture is sparse, the matching difficulty increases, requiring a lower texture threshold, a larger matching window, and a strictly set uniqueness ratio to avoid false matches. The level of scene depth difference reflects the variation of objects at different distances under the same line of sight. The greater the difference, the more isolated points or noise may be generated. Therefore, the speckle range can be increased to remove outliers, and the disparity search range can be expanded to cover targets at different depth levels. For scenes with small differences, these parameters can be reduced to improve computational efficiency. Scene motion affects the stability of image brightness and position. Large motion amplitude easily generates illumination and dynamic noise. Gradient features can be enhanced by increasing the preprocessing cutoff value to improve matching stability. For small motion amplitude, a lower cutoff value can be used to suppress extreme gradient noise. By quantifying these scene characteristics and establishing a mapping relationship with key parameters, the algorithm can automatically adjust parameters according to the actual scene, thereby improving the accuracy and robustness of stereo matching, while reducing human intervention and achieving adaptive optimization.

[0047] As shown in Table 1, texture complexity is related to the matching window size, texture threshold, and uniqueness ratio; scene depth difference level is related to the speckle range and parallax range; and scene motion degree is related to the preprocessing cutoff value. Therefore, key parameters can be adjusted based on scene characteristics. Furthermore, Table 1 also sets the constraint range for the key parameters.

[0048] Therefore, the key parameters of the dynamically adjusted BM algorithm include:

[0049] S4-1. Select any grayscale image and perform gradient analysis. Calculate the horizontal and vertical gradient magnitude matrices of the selected grayscale image using the Soble operator. Based on the horizontal and vertical gradient magnitude matrices, calculate the overall gradient magnitude matrix and normalize it to generate the texture complexity.

[0050] Specifically, the Soble operators for the selected grayscale image I along the x and y directions are defined by the following formula:

[0051] ;

[0052] in, Describes the Soble operator in the x-direction. This represents the Soble operator in the y-direction.

[0053] The horizontal gradient magnitude matrix of the grayscale image I along the horizontal and vertical directions is calculated using the Soble operator along the x and y directions. and vertical gradient magnitude matrix The corresponding formula is:

[0054] ;

[0055] in, This represents the convolution operation. , These represent the partial derivatives of the pixel values ​​of image I along the horizontal and vertical directions, respectively. and It is a matrix of dimension H×W, where W and H represent the width and height (number of pixels) of the image.

[0056] According to the formula:

[0057] ;

[0058] ;

[0059] Calculate the average gradient magnitude matrix ;in, , They represent the first The pixel and the The horizontal and vertical gradient magnitude matrices of a pixel. Represents the overall gradient magnitude matrix. This represents the summation function.

[0060] Due to the average gradient magnitude matrix When the value is greater than 50, the current scene has relatively rich texture. Using a scene with rich texture to calculate scene characteristics is beneficial for adjusting the key parameters of the subsequent BM algorithm, resulting in more accurate ranging results. Conversely, using a scene with sparse texture for key parameter adjustment will cause the subsequent ranging results to deviate from the actual distance. Therefore, the average gradient magnitude matrix is ​​determined. If the value exceeds 50, normalize the average gradient magnitude matrix to obtain the texture complexity. Conversely, if the image is not captured, it will be re-acquired.

[0061] Texture complexity The corresponding formula is:

[0062] ;

[0063] in, This represents the minimum value function.

[0064] S4-2. Calculate the pixel difference matrix of the two grayscale images and normalize it to generate the scene depth difference level.

[0065] Specifically, take a point in the grayscale image corresponding to the left view as... A point in the grayscale image of the right view is Calculate the pixel difference matrix of the left and right views. The corresponding formula is:

[0066] .

[0067] Based on the formula:

[0068] ;

[0069] Calculate the average pixel difference value Since the average pixel difference exceeds 70, the current scene is considered a depth scene, suitable for ranging, thus determining whether the average pixel difference exceeds 70. If it does, the average pixel difference is normalized to obtain the scene depth difference level; otherwise, it returns to S2 to re-acquire the image. Scene Depth Difference Level The corresponding formula is:

[0070] .

[0071] S4-3. Define two consecutive frames of the selected grayscale image, calculate the pixel difference between the frames and perform averaging and normalization to generate the scene motion level.

[0072] Specifically, scene motion is used to measure the difference between two consecutive frames, characterizing changes in scene lighting or other elements within the image. Therefore, the pixel value of a point on the selected grayscale image at the current moment (current frame) is defined as... The pixel value of a certain point in the previous frame's drawing image According to the formula:

[0073] ;

[0074] ;

[0075] Calculate average pixel difference , This represents the pixel difference between frames. Since an average pixel difference exceeding 30 indicates a significant change in the current scene over time, subsequent ranging results are considered more accurate to the actual distance. Therefore, the system determines whether the average pixel difference exceeds 30; if so, it adjusts the average pixel difference accordingly. Normalization is performed to obtain the scene motion level. Otherwise, return to S2 and re-acquire the image. The formula for normalization is:

[0076] .

[0077] S4-4. Based on texture complexity, scene depth difference level and scene motion degree, construct the mapping relationship between scene characteristics and key parameters, and adjust the gate in combination with feedback mechanism.

[0078] The adjustment of key parameters using the feedback mechanism includes:

[0079] S4-4-1. Based on texture complexity and in conjunction with the formula:

[0080] ;

[0081] ;

[0082] ;

[0083] Adjust the size of the matching window Texture threshold and uniqueness ratio ;in, Indicates texture complexity. This indicates a round-down operation. Represents the maximum value function;

[0084] Specifically, adjust the size of the matching window. Used for matching features in an image. A matching window that is too small will not be able to effectively match features; therefore, the minimum matching window size is 5. When texture complexity is low, most pixels and their surrounding pixels have small differences, making feature matching difficult. In this case, a larger matching window is used to provide sufficient matching information. When texture complexity is high, the image contains rich registrationable information, making feature matching easier. A smaller matching window is used to preserve edge details and further ensure... The value of is odd, which satisfies the requirements of the BM algorithm.

[0085] Texture threshold This is used to determine whether to perform pixel matching and controls the texture filter threshold. In this embodiment, the texture threshold is adjusted according to texture complexity. When the scene texture complexity is high, the texture threshold is increased to retain only the most reliable matching points and match in strong texture areas, reducing false matches. When the scene texture complexity is low, the texture threshold is decreased to allow weak texture matching and increase the matching point coverage; otherwise, too many areas would be filtered out, resulting in a large number of holes in the disparity map. Thus, the sensitivity of texture matching is controlled, making the depth map more closely resemble the actual image during stereo matching using the BM algorithm.

[0086] Uniqueness ratio To prevent false matches, the uniqueness ratio is checked to determine if the difference between the "best match" and the "second-best match" is large enough. In this embodiment, the uniqueness ratio is adaptively adjusted based on texture complexity. The richer the texture in the scene, the less ambiguity and the lower the matching difficulty. When the scene texture complexity is low, the calculated costs for many different disparity locations may be similar. In this case, the uniqueness ratio needs to be increased, and the best match must be "much better" than the second-best match to adapt to the current scene, filtering out ambiguous matching points. When the scene texture complexity is high, the cost difference between correct and incorrect matching points is large, so the judgment criteria are relaxed to retain more detailed features. Through this mechanism, false matches caused by texture repetition, weak texture, or noise can be effectively suppressed, thereby improving the accuracy and stability of the disparity map.

[0087] S4-4-2. Based on the level of scene depth difference, and in conjunction with the formula:

[0088] ;

[0089] ;

[0090] Adjust the spot range and parallax range ;in, Indicates the level of difference in scene depth;

[0091] Specifically, the range of spots The maximum permissible difference in depth discontinuities is defined to smooth small-scale depth abrupt changes and filter out isolated, potentially erroneous disparity regions. When the depth difference within a local region exceeds the SpeckleRange, it is usually accompanied by more occlusion and noise. The system treats this as noise or mismatch and discards it, thus avoiding the generation of a large number of isolated points or erroneous specks in the disparity map. The speckle range is then defined. By establishing a functional relationship with the scene depth difference level, adaptive adjustment is achieved, which effectively improves the continuity and stability of the depth map.

[0092] Parallax range This is used to limit the horizontal search range during stereo matching, restricting the BM algorithm to search within a specified disparity range. If the disparity range is too small, it will lead to the loss of matching between near and far objects; if the disparity range is too large, it will increase the computational load and may introduce additional errors. In this embodiment, the disparity range is adaptively adjusted according to the level of scene depth difference, so that it can cover the depth range of the main target while balancing computational efficiency and matching accuracy, thereby enhancing the system's depth perception capability. When the scene depth range is large, the disparity range is increased to ensure coverage from the nearest object to the farthest object; conversely, the disparity range is decreased to reduce computational load and memory usage, improve matching speed, and avoid wasting resources on useless disparity levels.

[0093] S4-4-3. Based on the degree of motion in the scene, and in conjunction with the formula:

[0094] ;

[0095] Adjusting the preprocessing cutoff value ;in, Indicates the depth of scene motion.

[0096] It's important to explain that in the BM algorithm, the image undergoes a pre-filter to enhance texture before calculating the cost. This requires a parameter to limit the maximum pixel intensity after filtering. Therefore, the pre-processing truncation value... This method truncates the brightness gradient of the input image before stereo matching to enhance matching stability and resist the effects of lighting changes. When there are moving objects or significant dynamic changes in lighting in the scene, a higher truncation value is set to preserve strong gradient features, thus ensuring the robustness of disparity calculation under complex lighting conditions. Conversely, under stable lighting conditions, a lower truncation value suppresses extreme gradient features, reducing noise interference and improving the consistency of matching results. This preprocessing truncation value... With the degree of scene movement By establishing a functional relationship, adaptive adjustment under different lighting conditions is achieved, thereby effectively improving the stability and accuracy of depth estimation. In this embodiment, when the scene lighting conditions are stable, i.e. When =0, the preprocessing cutoff value is taken. The base value is 15, which suppresses extreme gradient features and improves matching stability.

[0097] S4-4-4, Match window size Texture threshold Uniqueness ratio , range of spots Parallax range and preprocessing cutoff value Applying this to the BM algorithm yields the adjusted BM algorithm.

[0098] Step S4-4-4 is followed by S4-4-5, which determines whether the matching window size, texture threshold, uniqueness ratio, speckle range, disparity range, and preprocessing cutoff value are within the corresponding constraint range. If any key parameter is not within the corresponding constraint range, the image is re-acquired, i.e., the process returns to S2.

[0099] S5. Perform stereo matching on the two grayscale images using the adjusted BM algorithm, take any one of the grayscale images as a reference and combine it with the YOLO model to generate target detection information, and output the ranging result; the ranging result includes the category, position and distance information of the detected target.

[0100] The generated target detection information outputs ranging results, including:

[0101] S5-1. Perform stereo matching on two grayscale images using the adjusted BM algorithm, calculate the similarity and perform the optimal matching to generate a disparity map; generate a depth map based on the disparity map and the triangulation formula.

[0102] The generated depth map includes:

[0103] S5-1-1. Based on the preprocessing truncation value, the two grayscale images are enhanced by a pre-filter to improve texture information and suppress noise and illumination interference, resulting in a left-grayscale enhanced image and a right-grayscale enhanced image.

[0104] S5-1-2. Traverse each pixel of the left grayscale enhancement image, using each pixel as the center, and construct a pixel window based on the matching window size;

[0105] S5-1-3. Based on epipolar constraints, on the epipolar line of the right-grayscale enhancement image, the search for pixel windows is limited to the disparity range to obtain candidate pixel windows. The disparity range determined by scene characteristics enables the search to cover targets at different depth levels while controlling computational overhead, ensuring matching efficiency and accuracy.

[0106] Epipolar constraint refers to a condition in a binocular vision system with two cameras. If there is a point in the left camera image, its corresponding matching point in the right camera image must lie on a specific straight line (epidial line); conversely, a point in the right camera image will also have a corresponding matching point on its epipolar line in the left camera image. In other words, during stereo matching, the search range for corresponding points is narrowed from the entire image to a single epipolar line. This compresses the search range for matching points in the right view from the "full image" to "the row (or local region) containing the epipolar line," avoiding indiscriminate searching of the entire right view, significantly reducing the number of similarity calculations, and improving the efficiency of stereo matching.

[0107] S5-1-4. Calculate the similarity between each pixel window and different candidate pixel windows and calculate the matching cost. Select the initial best matching disparity with the minimum cost based on the texture threshold and construct the initial best matching disparity matrix.

[0108] Specifically, a pixel window in the left-grayscale enhancement image For example, compare it with each candidate pixel window in the right-grayscale enhancement image within the parallax range. As candidate matches, the matching cost between the current pixel window and different candidate pixel windows is calculated. The matching cost can be expressed as the sum of absolute differences or the sum of squared differences, yielding a similarity score for each candidate pixel window. A smaller similarity score indicates a greater similarity between the two windows, and a higher probability of a match. This indicates that the pixel window is related to the first pixel. The pixel distance offset from the candidate pixel window is called parallax.

[0109] The texture intensity (mean or variance of gradient) of the pixel window is compared with a texture threshold. If the texture intensity of the pixel window is lower than the texture threshold, the region is considered to lack sufficient features, and the candidate match is directly eliminated. This pixel window does not participate in the matching process, avoiding a large number of random false matches in low-texture or flat regions. If the texture intensity of the pixel window is higher than the texture threshold, the corresponding pixel is retained within the candidate matching range and proceeds to the next step of optimal match selection. Thus, pixel windows with texture intensity greater than the texture threshold are retained, and their similarity scores are preserved to form a candidate matching set.

[0110] In each pixel window, the minimum disparity is selected as the initial best matching disparity for the corresponding pixel, and an initial best matching disparity matrix is ​​constructed.

[0111] S5-1-5. Based on the spot range and uniqueness ratio, perform consistency checks and optimizations on the initial best matching disparity matrix to obtain the best matching disparity matrix and generate a disparity map.

[0112] It should be noted that the cost difference between the initial best-match disparity and the second-best-match disparity is calculated for each pixel window. It is then determined whether each cost difference is greater than the uniqueness ratio; if so, the corresponding candidate match is discarded; otherwise, the candidate match is retained. Finally, the second-best-match disparity matrix is ​​obtained and visualized to obtain the corresponding initial disparity map. The second-best-match disparity refers to the minimum disparity within a pixel window besides the initial best-match disparity.

[0113] The initial disparity map is examined for isolated or small-area outliers by checking the speckle range. Specks with abnormal depth continuity and areas smaller than a threshold are removed or filled. Next, a left-right consistency check is performed, matching the disparity of the left-grayscale enhancement image to that of the right-grayscale enhancement image, and comparing this with the disparity of the right-grayscale enhancement image to that of the left-grayscale enhancement image. Pixels with inconsistencies are considered outliers and are removed or corrected, resulting in the optimal matching disparity matrix and its disparity map.

[0114] S5-1-6. Process the disparity map using triangulation formulas to convert pixel coordinates into spatial coordinates and generate a depth map.

[0115] S5-2. Input any grayscale image into the YOLO model, use a convolutional network to extract multi-scale feature maps of the selected grayscale image, and combine an FPN or PANet structure with an object detection head to generate object detection information; the object detection information is the detection box and its position information, and its position information is [x Db ,y Db ,w,h,r]. Where, x Db y Db The value represents the center point of the detection box, w and h represent the width and height of the detection box, respectively, and r represents the angle of the detection box in the image.

[0116] Specifically, any grayscale image is input into the YOLO model. The model extracts multi-scale feature maps through a convolutional neural network and combines them with FPN or PANet structures to fuse and enhance the features, thereby improving the detection capability of small and multi-scale targets. Subsequently, the target detection head predicts the fused features to generate target detection information, including the position, size, angle of the detection box and the target category, so as to achieve accurate identification and localization of targets in the image and effectively support subsequent depth map fusion and accurate ranging.

[0117] S5-3. Using the center point of the detection box as a reference, read the corresponding depth value from the depth map and determine whether the depth value is valid. If the depth value is invalid, search for a valid depth point within the detection box and select the depth value closest to the center point as compensation, and output the ranging result. Otherwise, use the depth corresponding to the center point of the detection box as the ranging result.

[0118] like Figure 3 As shown, the process of obtaining the ranging result is as follows:

[0119] Using the center point of the detection box as a reference, the corresponding depth value is read from the depth map and it is determined whether the depth value is valid; if the depth value is in the range of [1, 10000], the depth value is valid, otherwise it is invalid.

[0120] When the depth value is valid, the depth corresponding to the center point of the detection box is used as the ranging result; when the depth value is invalid, the coordinates and depth values ​​of all valid depth values ​​within the detection box are obtained, the distance between each point and the center point of the detection box is calculated, and the depth value of the point closest to the center of the detection box is taken as the target depth value and used as the ranging result.

[0121] It should be noted that, Figure 1 The execution entity of the corresponding method shown can be a software and / or hardware device. The execution entity of this application can include, but is not limited to, at least one of the following: user equipment, network equipment, etc. User equipment can include, but is not limited to, computers, smartphones, personal digital assistants (PDAs), and the aforementioned electronic devices. Network equipment can include, but is not limited to, a single network server, a server group consisting of multiple network servers, or a cloud based on cloud computing consisting of a large number of computers or network servers. Cloud computing is a type of distributed computing, consisting of a super virtual computer composed of a group of loosely coupled computers. This embodiment does not impose any limitations on this.

[0122] like Figure 4 As shown, a scene-feature-based parameter adaptive matching binocular ranging system includes:

[0123] The pixel-space mapping relationship construction module is used to perform geometric calibration of the stereo camera, obtain the camera intrinsic and extrinsic parameter matrices, and establish the mapping relationship between pixel coordinates and actual spatial coordinates.

[0124] The left and right view real-time acquisition module is used to synchronously acquire the left and right views of the current scene in real time through a geometrically calibrated binocular camera;

[0125] The stereo correction module is used to perform stereo correction on the left and right views and generate corresponding grayscale images through image conversion.

[0126] The scene characteristic calculation module is used to evaluate the scene characteristics of the current scene based on any grayscale image;

[0127] The BM algorithm adaptive adjustment module is used to dynamically adjust the key parameters of the BM algorithm based on scene characteristics;

[0128] The stereo matching module is used to perform stereo matching on two grayscale images using the adjusted BM algorithm to generate a depth map;

[0129] The target detection information generation module is used to generate target detection information by taking any grayscale image as a reference and combining it with the YOLO model.

[0130] The ranging result calculation module is used to calculate the ranging result based on the depth map and target detection information.

[0131] The scene characteristic calculation module includes:

[0132] The texture complexity module is used to calculate the horizontal and vertical gradient magnitude matrices of the selected grayscale image using the Soble operator, and generate the texture complexity.

[0133] The scene depth difference level module is used to calculate the pixel difference matrix of two grayscale images and normalize it to generate the scene depth difference level.

[0134] The scene motion level module is used to define two consecutive frames of the selected grayscale image, calculate the pixel differences between the frames, and perform averaging and normalization to generate the scene motion level.

[0135] It should be noted that the specific methods by which each module performs operations in the system described in the above embodiments have been described in detail in the embodiments related to the device, and will not be elaborated here.

[0136] In summary, by introducing a scene characteristic calculation module, this system can adaptively adjust the key parameters of the BM algorithm based on texture complexity, depth difference level, and scene motion, thereby maintaining stable matching results under different lighting, texture, and dynamic environments, and improving ranging accuracy and robustness. Simultaneously, by fusing target detection information generated by the YOLO model with the depth map, it achieves integrated target recognition and ranging, reducing manual intervention, improving automation, and making the system more adaptable and practical in complex scenarios such as intelligent monitoring, autonomous driving, and robot perception.

[0137] Example 2:

[0138] A cable (target) is photographed in a specific scene using a stereo camera, and the two captured images are used as experimental images. This method is then used to process any one of the experimental images to calculate the distance between the cable and the stereo camera.

[0139] The scene characteristics of the experimental image were calculated, and the corresponding values ​​for texture complexity, scene depth difference level, and scene motion degree were 0.11, 0.08, and 0.02, respectively. Based on the calculated scene characteristics, the key parameters of the BM algorithm were dynamically adjusted, and the results are shown in Table 2.

[0140] Table 2

[0141]

[0142] As shown in Table 2, each key parameter conforms to the constraint range of the key parameters in Table 1, thus allowing the key parameters to be combined and applied to the BM algorithm.

[0143] Using the adjusted BM algorithm, stereo matching is performed on the corresponding grayscale images of the two experimental images to obtain the corresponding depth maps, such as... Figure 5 As shown. Based on this depth map and the experimental image, a positional relationship map is obtained, as shown below. Figure 6 As shown, the experimental images are processed using the YOLO model to obtain the corresponding target detection information (the detection boxes corresponding to the cables). Based on the target detection information and the depth map, the ranging result is generated, as shown below. Figure 7 As shown.

[0144] Depend on Figure 5 and Figure 6 It is known that by using the BM algorithm with scene characteristic adjustments for stereo matching and depth transformation, images reflecting the depth of the real scene can be generated. Even with slight noise, the depth levels of the scene can still be captured, providing a reliable data foundation for "target-distance" correlation. Furthermore, Figure 6 The mapping relationship between the mid-depth information and the experimental image is clear, indicating that the depth value can accurately match the pixels / regions of the original image, ensuring that the depth information can be associated with the target "one-to-one" during subsequent target detection and depth fusion, thus avoiding matching bias.

[0145] Depend on Figure 7 It is evident that the detection frame precisely surrounds the cable, and the output distance measurement result is clearly "1.94m," proving that the solution can effectively integrate the "numerical information of the depth map" with the "spatial positioning of the target detection," achieving accurate distance measurement of a specific target. Simultaneously, through visual annotation (green box, text result), the results are intuitively displayed, verifying the reliability and practicality of the entire "stereo matching → depth map generation → target distance measurement" process of this invention. Figure 7 In this context, cabel represents cable, meaning the target category is cable; 8.4deg indicates that the angle r of the detection box in the image is 8.4°.

[0146] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

[0147] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A parameter adaptive matching binocular ranging method based on scene features, characterized in that, include: Perform geometric calibration on the stereo camera, obtain the camera intrinsic and extrinsic parameter matrices, and establish the mapping relationship between pixel coordinates and actual spatial coordinates; The left and right views of the current scene are acquired in real time using a geometrically calibrated binocular camera. Perform stereo correction on the left and right views, and generate corresponding grayscale images through image conversion; The scene characteristics of the current scene are evaluated based on any grayscale image, and the key parameters of the BM algorithm are dynamically adjusted. The key parameters include matching window size, texture threshold, uniqueness ratio, speckle range, disparity range, and preprocessing cutoff value. The scene characteristics include texture complexity, scene depth difference level, and scene motion degree. The adjusted BM algorithm is used to perform stereo matching on two grayscale images. One of the grayscale images is used as a reference and combined with the YOLO model to generate target detection information, and the ranging result is output. The ranging results include the category, location, and distance information of the detected target; The generated target detection information outputs ranging results, including: The modified BM algorithm is used to perform stereo matching on two grayscale images, calculate the similarity and perform the optimal matching to generate a disparity map; based on the disparity map and combined with the triangulation formula, a depth map is generated. Any grayscale image is input into the YOLO model, and a multi-scale feature map of the input grayscale image is extracted using a convolutional network. This feature map is then combined with an FPN or PANet structure and an object detection head to generate object detection information. The object detection information consists of a detection box and its location information. Using the center point of the detection box as a reference, the corresponding depth value is read from the depth map and it is determined whether the depth value is valid; if the depth value is in the range of [1, 10000], the depth value is valid, otherwise it is invalid. When the depth value is valid, the depth corresponding to the center point of the detection box is taken as the ranging result; when the depth value is invalid, the coordinates and depth values ​​of all points with valid depth values ​​within the detection box are obtained, the distance between each point and the center point of the detection box is calculated, and the depth value of the point closest to the center of the detection box is taken as the target depth value and used as the ranging result. The generated depth map includes: Based on the preprocessed truncation value, the two grayscale images are enhanced by a pre-filter to obtain a left-grayscale enhanced image and a right-grayscale enhanced image; Iterate through each pixel of the left grayscale enhancement image, using each pixel as the center, and construct a pixel window based on the matching window size; Based on epipolar constraints, a pixel window is searched within the disparity range along the epipolar line of the right-grayscale enhancement image to obtain a candidate pixel window; The similarity between each pixel window and different candidate pixel windows is calculated and the matching cost is calculated. The initial best matching disparity with the minimum cost is selected according to the texture threshold, and the initial best matching disparity matrix is ​​constructed. Based on the spot range and uniqueness ratio, the initial best matching disparity matrix is ​​checked for consistency and optimized to obtain the best matching disparity matrix and generate a disparity map. By processing the disparity map using triangulation formulas, pixel coordinates are converted into spatial coordinates to generate a depth map.

2. The scene feature-based parameter adaptive matching binocular ranging method according to claim 1, characterized in that, The process of establishing the mapping relationship between pixel coordinates and actual spatial coordinates includes: Acquire calibration images, perform single-target intrinsic parameter calibration of the stereo camera using Zhang Zhengyou calibration method or OpenCV calibration tool, and obtain the corresponding intrinsic parameter matrix; Match the corner coordinates of the stereo camera on the calibration image, calculate the rotation matrix and translation vector of the stereo camera, and construct the extrinsic parameter matrix of the stereo camera; Based on two intrinsic parameter matrices and two extrinsic parameter matrices, disparity and spatial depth are calculated by calibrating the pixel coordinates on the image, and a mapping relationship between pixel coordinates and actual spatial coordinates is established.

3. The scene feature-based parameter adaptive matching binocular ranging method according to claim 1, characterized in that, The key parameters of the dynamically adjusted BM algorithm include: Arbitrarily select a grayscale image and perform gradient analysis. Calculate the horizontal and vertical gradient magnitude matrices of the selected grayscale image using the Soble operator. Based on the horizontal and vertical gradient magnitude matrices, calculate the overall gradient magnitude matrix and normalize it to generate the texture complexity. Calculate the pixel difference matrix of the two grayscale images and normalize it to generate the scene depth difference level; Define two consecutive frames of the selected grayscale image, calculate the pixel difference between the frames, and perform averaging and normalization to generate the scene motion level; Based on texture complexity, scene depth difference level, and scene motion degree, a mapping relationship between scene characteristics and key parameters is constructed, and the key parameters are adjusted in combination with a feedback mechanism.

4. The scene feature-based parameter adaptive matching binocular ranging method according to claim 3, characterized in that, The adjustment of key parameters using the feedback mechanism includes: Based on texture complexity, and combined with the formula: ； ； ； Adjust the size of the matching window Texture threshold and uniqueness ratio ;in, Indicates texture complexity. This indicates a round-down operation. Represents the maximum value function; Based on the level of scene depth difference, and combined with the formula: ； ； Adjust the spot range and parallax range ;in, Indicates the level of difference in scene depth; Based on the degree of motion in the scene, and combined with the formula: ； Adjusting the preprocessing cutoff value ;in, Indicates the degree of scene motion Match window size Texture threshold Uniqueness ratio , range of spots Parallax range and preprocessing cutoff value Applying this to the BM algorithm yields the adjusted BM algorithm.

5. A scene-feature-based parameter adaptive matching binocular ranging system, used to implement the scene-feature-based parameter adaptive matching binocular ranging method according to any one of claims 1 to 4, characterized in that, include: The pixel-space mapping relationship construction module is used to perform geometric calibration of the stereo camera, obtain the camera intrinsic and extrinsic parameter matrices, and establish the mapping relationship between pixel coordinates and actual spatial coordinates. The left and right view real-time acquisition module is used to synchronously acquire the left and right views of the current scene in real time through a geometrically calibrated binocular camera; The stereo correction module is used to perform stereo correction on the left and right views and generate corresponding grayscale images through image conversion. The scene characteristic calculation module is used to evaluate the scene characteristics of the current scene based on any grayscale image; The BM algorithm adaptive adjustment module is used to dynamically adjust the key parameters of the BM algorithm based on scene characteristics; The stereo matching module is used to perform stereo matching on two grayscale images using the adjusted BM algorithm to generate a depth map; The target detection information generation module is used to generate target detection information by taking any grayscale image as a reference and combining it with the YOLO model. The ranging result calculation module is used to calculate the ranging result based on the depth map and target detection information.

6. The scene feature-based parameter adaptive matching binocular ranging system according to claim 5, characterized in that, The scene characteristic calculation module includes: The texture complexity module is used to calculate the horizontal and vertical gradient magnitude matrices of the selected grayscale image using the Soble operator, and generate the texture complexity. The scene depth difference level module is used to calculate the pixel difference matrix of two grayscale images and normalize it to generate the scene depth difference level. The scene motion level module is used to define two consecutive frames of the selected grayscale image, calculate the pixel differences between the frames, and perform averaging and normalization to generate the scene motion level.

Images