A size feature-oriented binocular vision re-measurement guiding method

Abstract

The application relates to the technical field of binocular vision measurement, and discloses a binocular vision remeasurement guiding method for size features. The method comprises the following steps: collecting a left image and a right image of a measured object through a binocular vision system, and generating a three-dimensional point cloud of the measured object through a binocular vision algorithm according to the left image and the right image; determining at least one to-be-measured size and a corresponding size feature, wherein the size feature is a geometric feature used for calculating the to-be-measured size; calculating a size confidence degree of each to-be-measured size according to the reliability of each point in a size feature region of each to-be-measured size; the size feature region is a set of pixel points corresponding to the size feature in the left image and the right image, and a set of three-dimensional points corresponding to the size feature in the three-dimensional point cloud; performing remeasurement in the case that the size confidence degree does not meet a confidence degree requirement, until the size confidence degree of the to-be-measured size meets the confidence degree, and outputting a measurement result of the to-be-measured size. The method can improve the reliability of binocular vision size measurement results.

Description

Technical Field

[0001] This application relates to the field of binocular vision measurement technology, and more specifically, to a binocular vision retest guidance method oriented towards size features. Background Technology

[0002] Binocular vision measurement technology acquires images of the object being measured simultaneously using two cameras, left and right. It then recovers the three-dimensional coordinates of the object based on the parallax relationship between the left and right images, and calculates the geometric dimensions of the object (i.e., the measurement result). It has advantages such as being non-contact, relatively low cost, and highly adaptable, and has been widely used in industrial scenarios such as industrial parts inspection, robot positioning, 3D reconstruction, assembly measurement, and rapid on-site inspection.

[0003] In practical applications, binocular vision measurement results are easily affected by a variety of factors. For example, the surface of the object being measured may have weak texture, strong reflection, local occlusion, blurred edges, uneven lighting, excessive shooting distance, and left and right image matching conflicts. These problems can lead to sparse local point clouds, increased parallax error, unstable edge positioning, or unreliable local size calculations.

[0004] To improve the accuracy of binocular vision measurement results, existing binocular vision measurement methods typically focus on improving the accuracy of disparity calculation, optimizing the quality of 3D point clouds, or outputting overall measurement results. Some methods can generate pixel-level or point cloud-level confidence information to determine whether the depth values ​​of certain regions are reliable.

[0005] However, in actual dimensional measurement tasks, users are usually not truly concerned with the reliability of the entire depth map, but rather with the reliability of a specific dimension. Furthermore, even if certain areas with low confidence are identified, existing methods often simply discard low-quality points, indicate measurement failure, or require re-acquiring the entire image. For non-professional operators, it is difficult to identify low-quality points and to promptly determine the specific cause of low-confidence measurement results. Therefore, when re-acquiring the entire image for re-measurement, it is difficult to determine how to adjust the shooting angle, distance, exposure, lighting, or measurement position. Moreover, if the entire object is rescanned every time a measurement result is low-confidence, it reduces dimensional measurement efficiency and may introduce new point cloud registration errors.

[0006] Therefore, it is necessary to provide a binocular vision-guided retesting method for dimensional feature measurement tasks. This method not only evaluates the quality of the depth map, but also judges the confidence level of the endpoints, edges, hole contours, fitting planes, or local contour segments corresponding to the dimension to be measured. This addresses the problems in existing technologies, such as the difficulty in judging the reliability of local dimensional measurement results, the difficulty in determining the specific causes of low-confidence measurement results, and the lack of retesting guidance schemes. Summary of the Invention

[0007] To address the issue of unstable local dimension measurement results in binocular vision-based dimension measurement, this invention provides a binocular vision retest guidance method oriented towards dimension features. This method uses the dimension features of the dimension to be measured as the core evaluation object to determine low confidence levels. If a low confidence level is determined, the cause of the low confidence is identified and retest guidance is provided. This allows for more accurate identification of low-confidence measurement results in local dimensions such as aperture, aperture distance, edge distance, step height, and contour deviation, thereby improving the reliability of binocular vision-based dimension measurement results.

[0008] To achieve the above objectives, according to a first aspect of the present invention, a binocular vision retest guidance method oriented towards size features is provided, the method comprising: The left and right images of the object under test are acquired through a binocular vision system, and a three-dimensional point cloud of the object under test is generated based on the left and right images using a binocular vision algorithm. Identify at least one dimension to be measured and its corresponding dimensional feature, wherein the dimensional feature is a geometric feature used to calculate the dimension to be measured; Based on the reliability of each point within the size feature region of each size to be measured, the size confidence level of each size to be measured is calculated; the size feature region is the set of pixels corresponding to the size feature in the left and right images, and the set of three-dimensional points corresponding to the size feature in the three-dimensional point cloud; If the size confidence level does not meet the confidence level requirement, a retest is performed: the size confidence level of each size to be tested is recalculated based on the reliability of each point within the size characteristic area of ​​each size to be tested, until the size confidence level of the size to be tested meets the confidence level requirement; If the confidence level of the dimensions meets the requirements, the measurement results of the dimensions to be measured are calculated and output based on the dimensional characteristics.

[0009] Furthermore, the dimensional features include at least one of the following: the endpoints, edges, hole profiles, fitting planes, step surfaces, or local profile segments corresponding to the dimensions.

[0010] Furthermore, the confidence level requirement is that the size confidence level is not lower than the confidence level threshold. The aforementioned binocular vision retest guidance method for size features also includes measuring the size to be measured of several calibration samples with known real sizes using a binocular vision system to obtain the measured size of each calibration sample; for each calibration sample, calculating the absolute value of the difference between its real size and measured size to obtain the measurement error corresponding to each calibration sample; calculating the size confidence level of the size to be measured for each calibration sample; forming a confidence level and error sample set based on the size confidence levels and corresponding measurement errors of several calibration samples; setting a candidate confidence level threshold, statistically analyzing all calibration samples in the confidence level and error sample set that satisfy the size confidence level not lower than the candidate confidence level threshold, and calculating the proportion of those samples whose measurement error does not exceed the allowable measurement error; adjusting the candidate confidence level threshold, and determining the smallest candidate confidence level threshold whose proportion is not lower than the preset reliable output probability as the confidence level threshold.

[0011] Furthermore, the aforementioned binocular vision retesting guidance method oriented towards size features also includes calculating the reliability of each point within the size feature region of each measured size by weighted summation based on at least two of the following: left-right matching consistency, local texture validity, edge sharpness, brightness validity, disparity validity, or point cloud local stability. Left-right matching consistency is used to determine whether the correspondence between the left and right images is stable. Local texture validity is used to determine whether the size feature region has a matchable texture that meets the set requirements. Edge sharpness is used to determine whether the boundary or contour of the measured object is clear. Brightness validity is used to suppress overexposed, underexposed, or strongly reflective areas in the left and right images. Disparity validity is used to determine whether the current disparity falls within the set disparity range. Point cloud local stability is used to describe the degree of fluctuation in the three-dimensional coordinates of the current point's neighborhood.

[0012] Furthermore, based on the reliability of each point within the dimensional feature region of each dimension to be measured, the dimensional confidence level of each dimension to be measured is calculated, including calculating the average confidence level of the dimensional feature region based on the reliability of each point within the dimensional feature region; and determining the dimensional confidence level of each dimension to be measured based on the average confidence level of the dimensional feature region.

[0013] Furthermore, based on the average confidence score of the size feature region, the size confidence score of each size to be measured is determined, including obtaining at least one of the following: geometric integrity confidence score, geometric fitting stability confidence score, and size calculation stability confidence score. The geometric integrity confidence score is used to describe whether the key features required for the size to be measured are complete; the geometric fitting stability confidence score is used to describe the fitting error of the size features; and the size calculation stability confidence score is used to describe whether the size results calculated by different local sampling subsets within the current size feature region are consistent. The size confidence score of each size to be measured is calculated by weighted summation of at least one of the geometric integrity confidence score, geometric fitting stability confidence score, and size calculation stability confidence score and the average confidence score of the size feature region.

[0014] Furthermore, if the dimensional confidence level does not meet the confidence level requirement, a retest is performed: The dimensional confidence level of each tested dimension is recalculated based on the reliability of each point within the dimensional characteristic region of each tested dimension until the dimensional confidence level of the tested dimension meets the confidence level requirement. This includes identifying the set of low-confidence causes when the dimensional confidence level does not meet the confidence level requirement; calculating the evaluation function for each type of low-confidence cause in the low-confidence cause set to form a set of low-confidence influence parameters; identifying the type of low-confidence cause with the largest evaluation function as the most important low-confidence cause; and performing a retest based on the set of low-confidence influence parameters: recalculating the dimensional confidence level of each tested dimension based on the reliability of each point within the dimensional characteristic region of each tested dimension until the dimensional confidence level of the tested dimension meets the confidence level requirement.

[0015] Furthermore, the set of causes for low confidence includes at least one of the following: occlusion-related low confidence, brightness anomaly-related low confidence, weak texture-related low confidence, insufficient parallax-related low confidence, blurred edge-related low confidence, left-right matching conflict-related low confidence, or local point cloud sparseness-related low confidence.

[0016] Further, based on the set of low-confidence impact parameters, a retest is performed: the size confidence of each size to be measured is recalculated based on the reliability of each point within the size feature region of each size to be measured until the size confidence of the size to be measured meets the confidence requirement. This includes generating a set of candidate retest actions based on the trigger function of the evaluation function of at least one type of low-confidence cause in the set of low-confidence impact parameters and the corresponding retest action generation rules. The retest action generation rules are a set of size features, size feature geometric states, and current acquisition states. The size feature geometric states are used to describe the position and orientation of the size to be measured in space. The current acquisition states include a set representing the current camera pose, current exposure parameters, current fill light parameters, and current focus state. A comprehensive cost evaluation is performed on each candidate retest action in the candidate retest action set, and the candidate retest action with the lowest comprehensive cost is determined as the retest guidance action. The retest operation is performed according to the retest guidance action: the size confidence of each size to be measured is recalculated based on the reliability of each point within the size feature region of each size to be measured until the size confidence of the size to be measured meets the confidence requirement.

[0017] Furthermore, the evaluation function for occlusion-related low confidence is:

[0018] in, This represents the number of features detected within the size-specific region of the left image. This represents the number of features that can be matched in both the left and right images. When the low confidence level is due to occlusion, the size feature region is partially invisible or has missing matching in the left and right images. The goal of the retest is to change the viewing direction so that the originally occluded region enters the binocular common visibility range. The corresponding retest action generation rules include at least one of changing the viewing direction, moving the camera laterally, or rotating the object being tested.

[0019] In summary, compared with the prior art, the above-described technical solutions conceived by this invention can achieve the following beneficial effects: (1) The present invention provides a binocular vision retest guidance method oriented towards size features. Taking the size features of the size to be measured as the core evaluation object, a size confidence evaluation mechanism is established for specific measurement features such as endpoints, edges, hole contours, fitting planes, step surfaces or local contour segments. This avoids judging the reliability of measurement results based solely on the quality of the entire image or the overall point cloud quality. As a result, it can more accurately identify low-confidence measurement results in local dimensions such as aperture, hole distance, edge distance, step height, and contour deviation, thereby improving the reliability of binocular vision size measurement results.

[0020] (2) The binocular vision retesting guidance method for size features provided by the present invention can further determine the reasons for low confidence when the size confidence does not meet the confidence requirements, and generate targeted retesting guidance actions according to different reasons such as occlusion, reflection, weak texture, insufficient parallax, blurred edges, left and right matching conflict or local point cloud sparseness. The retesting operation is performed according to the retesting guidance actions, thereby improving the efficiency of on-site measurement, reducing the operation difficulty for non-professionals, and enhancing the adaptability in application scenarios such as handheld measurement, robot measurement and production line online inspection. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 A flowchart illustrating a binocular vision retesting guidance method oriented towards size features, provided for one embodiment of this application; Figure 2 This is a schematic diagram of a binocular vision retesting guidance method oriented towards size features, provided for another embodiment of this application. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0024] The terms "first," "second," "third," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or apparatuses.

[0025] like Figure 1As shown, a binocular vision retest guidance method oriented towards size features is provided. This method can be executed by a terminal or by a server communicating with the terminal via a network. The terminal can be, but is not limited to, various personal computers, laptops, smartphones, tablets, etc. The server can be a standalone server or a server cluster consisting of multiple servers. Taking the application of this method to a terminal as an example, the following steps are included: Step 101: Acquire the left and right images of the object under test using a binocular vision system, and generate a 3D point cloud of the object under test based on the left and right images using a binocular vision algorithm.

[0026] Step 102: Determine at least one dimension to be measured and its corresponding dimensional features, wherein the dimensional features are geometric features used to calculate the dimension to be measured.

[0027] The dimensional features include at least one of the following: the endpoints, edges, hole profiles, fitting planes, step surfaces, or local profile segments corresponding to the dimensions.

[0028] Step 103: Calculate the size confidence of each size to be measured based on the reliability of each point within the size feature region of each size to be measured; the size feature region is the set of pixels corresponding to the size feature in the left and right images, and the set of three-dimensional points corresponding to the size feature in the three-dimensional point cloud.

[0029] Step 104: If the size confidence does not meet the confidence requirement, perform a retest: recalculate the size confidence of each size under test based on the reliability of each point within the size characteristic area of ​​each size under test until the size confidence of the size under test meets the confidence requirement. Step 105: If the size confidence level meets the confidence level requirements, calculate and output the measurement results of the size to be measured based on the size characteristics.

[0030] The measurement results of the dimension to be measured include at least one of the following: hole diameter, hole spacing, edge distance, step height, or profile deviation.

[0031] The aforementioned binocular vision remeasurement guidance method oriented towards size features takes the size features of the dimension to be measured as the core evaluation object. It establishes a size confidence evaluation mechanism for specific measurement features such as endpoints, edges, hole contours, fitting planes, step surfaces, or local contour segments. This avoids judging the reliability of measurement results solely based on the quality of the entire image or the overall point cloud. As a result, it can more accurately identify low-confidence measurement results in local dimensions such as aperture, hole distance, edge distance, step height, and contour deviation, thereby improving the reliability of binocular vision size measurement results.

[0032] In one embodiment, such as Figure 2As shown, a binocular vision retesting guidance method oriented towards size features first determines the geometric features corresponding to the size to be measured based on the measurement task. It then comprehensively considers factors such as left-right matching consistency, local texture effectiveness, edge sharpness, brightness effectiveness, disparity effectiveness, point cloud local stability, geometric structure integrity, geometric fitting stability, and size calculation stability to conduct a size confidence assessment. When the size confidence of a certain size meets the confidence requirement (i.e., not lower than the confidence threshold), the measurement result of that size is directly output (also called the size parameter to be measured, referring to the part size parameters that the binocular vision system needs to measure, such as the diameter of a cylinder or hole, the length of a cylinder, the side length of a plane, the height of a step, etc.). When the size confidence is lower than the confidence threshold, the main reasons for the low confidence are further determined, and corresponding retesting guidance actions are generated based on the different main reasons. After the retest is completed, only the low-confidence size feature regions are locally updated and merged to update the size parameters to be measured, and the size confidence is evaluated again. If it is still below the confidence threshold, the above steps are repeated until the confidence requirement is met; otherwise, the size parameters to be measured are output. The specific steps are as follows: Step 1: Initial binocular measurement and 3D point cloud generation.

[0033] Suppose that the binocular camera simultaneously acquires the left and right images of the object being measured, respectively. and ,in This represents the pixel coordinates of the image. After binocular calibration and epipolar correction, corresponding pixels of the same spatial point in the left and right images lie on the same epipolar line. Let the disparity of a certain pixel in the left image be . Then the depth corresponding to this pixel is:

[0034] Where f is the camera focal length and B is the baseline length of the binocular camera.

[0035] like Let the principal point coordinates of the camera be the coordinates. Then, the corresponding 3D coordinates of this pixel can be represented as:

[0036]

[0037]

[0038] When the number of valid 3D points in the initial point cloud is At this time, the initial three-dimensional point cloud of the object being measured can be obtained as follows:

[0039] The initial point cloud can be used for the first size calculation, but this method does not directly use the result as the final result. Instead, it continues to evaluate the reliability of the size features of the size to be measured (hereinafter referred to as the size features to be measured).

[0040] Step 2: Definition and modeling of the dimensional features to be measured.

[0041] Suppose the current measurement task includes One size to be measured For any dimension to be measured, It can create a corresponding size feature object. ,in, This represents the geometric feature corresponding to the dimension to be measured. The size feature region refers to the supporting region of the size feature in the image and point cloud (i.e., the set of pixels corresponding to the size feature in the left and right images, and the set of 3D points corresponding to the size feature in the 3D point cloud). Indicates the geometric state of dimensional features. This represents a measurement function that calculates dimensional values ​​from geometric features. From the above analysis, it can be seen that the dimension to be measured... It is not an isolated value, but is determined by its corresponding feature region, geometric model and measurement function.

[0042] For example, in length measurement, if the length is divided into two endpoints... and Decision, then ,in, For the edge region connecting the two endpoints, its size feature region consists of the neighborhoods of the two endpoints in the left and right images, the neighborhood of the edge, and the corresponding 3D point cloud region reconstructed from these image regions. Therefore, the size to be measured is:

[0043] For aperture measurement, the dimensional feature is the set of points on the aperture profile, i.e. The size characteristic region is a ring-shaped neighborhood near the hole boundary or hole outline, including the hole outline pixels in the left and right images, a neighborhood of a certain width inside and outside the hole outline, and the corresponding 3D point cloud region reconstructed from these image regions. After performing least-squares circle fitting on the hole outline points, the least-squares circle radius is obtained. Then the diameter of the hole to be measured is:

[0044] For step height measurement, the dimensional characteristics are two local planes, namely... Let among them The expression is , The expression is Its size feature regions are two local planar regions involved in height calculation, namely the upper step surface region and the lower step surface region, as well as the corresponding 3D point cloud region reconstructed from these image regions. If we consider... If the plane is positioned along a specified height, then the step height, the dimension to be measured, can be expressed as:

[0045] Based on the modeling method for the dimensions to be measured described above, different types of dimensions to be measured can be uniformly expressed as:

[0046] Subsequent confidence assessments, low confidence determinations, and retest guidance all revolve around... Expand.

[0047] It should be noted that the order of steps 1 and 2 above can also be that step 2 is performed first and then step 1. The order of steps 1 and 2 in this application is not limited by the described order of actions.

[0048] Step 3: Calculate the basic reliability of the size feature region.

[0049] After generating the initial point cloud, the basic reliability of each point within the size feature region is calculated first. For the size feature region... For any point or pixel within the range, its basic reliability model is defined as:

[0050] in, To ensure consistency in left and right matching, For local texture validity, For edge sharpness, For brightness effectiveness, For parallax effectiveness, For the local stability of point clouds, These are the corresponding weights, and The specific meanings and expressions of each reliability component are as follows: (1) Left-right matching consistency Used to determine the midpoint of left and right images Whether the correspondence is stable is specifically expressed as:

[0051] in, The disparity from the left image to the right image. The disparity is obtained by reverse verification from the right image to the left image; This is a matching error adjustment parameter used to characterize the impact of left and right image matching errors on size reliability. It is generally set to a value of 0.2 to 1. The larger the value, the smaller the impact of left and right image matching errors on reliability.

[0052] (2) Local texture effectiveness The method used to determine whether a region has enough matching textures can be represented as:

[0053] in, Midpoint of the left and right images Neighborhood image grayscale variance; This is a texture adjustment parameter, and its value can be determined based on the image's grayscale dynamic range and camera noise level. It is generally set to 5~50. The larger the value, the smaller the impact of weak texture areas on reliability.

[0054] (3) Edge sharpness Used to determine whether the boundary or contour to be measured is clear, it can be represented as:

[0055] in, Midpoint of the left and right images The image gradient magnitude; This is an edge adjustment parameter used to characterize the degree of impact of edge blur on dimensional reliability. Its value can be determined based on imaging resolution, lens depth of field, and edge gradient statistics. It is generally 1 to 8. The larger the value, the smaller the impact of edge blur on reliability.

[0056] (4) Brightness effectiveness To suppress overexposed, underexposed, or highly reflective areas in the left and right images, it can be represented as:

[0057] in, and These are the upper and lower limits of the effective brightness range, respectively; This is the brightness anomaly penalty coefficient, which can be determined based on the degree of brightness deviation in overexposed and underexposed samples. It is generally set between 2 and 10. The larger the value, the smaller the impact of brightness on reliability.

[0058] (5) Parallax effectiveness Used to determine the midpoint of the left and right images. Does the current parallax fall within the range suitable for measurement?

[0059] in, and These are the current allowed lower and upper limits of parallax, respectively; This is the parallax abnormality penalty coefficient, which can be determined based on the depth error after the parallax exceeds the effective range. It is generally taken as 0.2 to 1.0. The larger the value, the smaller the impact of parallax on reliability.

[0060] (6) Local stability of a point cloud is used to describe the degree of fluctuation of the three-dimensional coordinates of the neighborhood of the point, and can be expressed as:

[0061] in, Midpoint of a 3D point cloud Spatial standard deviation of neighborhood point cloud; This is a point cloud stability adjustment parameter, which can be determined based on local point cloud noise, fitting residuals, and allowable dimensional errors. In precision measurements, it is generally taken as 0.01~0.2, while in ordinary field measurements, it is generally taken as 0.1~1.0. The larger the value, the smaller the impact of local point cloud stability on reliability.

[0062] Thus, points within each size feature region Each has a corresponding basic reliability This provides a data foundation for subsequent size confidence calculations.

[0063] Step 4: Calculate the size confidence level.

[0064] The basic reliability mentioned above reflects whether a local point or pixel is reliable. In order to characterize whether a certain size is reliable, the size confidence is further calculated based on the overall structure of the size characteristics, on the basis of the basic reliability.

[0065] For the dimension to be measured Its size confidence level is defined as:

[0066] in, The average confidence level for the size feature region. For the sake of geometric structural integrity, For geometric fit stability, For dimensional stability calculation, These are the weighting coefficients, and The specific meanings and calculation formulas of each confidence component are as follows: (1) The mathematical expression for the average confidence level of the size feature region is:

[0067] in, For point Weighting of dimensional measurements based on their importance. Higher weights can be assigned to critical areas such as edges, hole profiles, and endpoints.

[0068] (2) Confidence level of geometric structural integrity Used to describe whether the dimensional features of the dimension to be measured are complete.

[0069] For example, for aperture measurement, the confidence level of its geometric integrity can be expressed based on the effective aperture profile coverage angle:

[0070] in, The angular range covered by the effective hole profile points.

[0071] For step height measurement, the confidence level of its geometric integrity can be represented by the number of valid points on the two planes:

[0072] in, and These represent the number of valid points on the two step planes, respectively. and These represent the minimum number of points required to complete a reliable plane fit.

[0073] (3) Confidence of geometric fit stability Fitting error used to describe the dimensional characteristics of the dimension to be measured.

[0074] For example, for aperture measurement, the confidence level of its geometric fit stability can be expressed as:

[0075] in, The average residual from the hole profile points to the fitted circle; The fitting error adjustment parameter can be determined based on the fitting residuals of the size characteristics. For precision measurements, it is generally taken as 0.02–0.15, and for ordinary field measurements, it is generally taken as 0.1–1.2. The larger the value, the smaller the influence of the fitting error on the confidence level.

[0076] For step height measurement, the confidence level of its geometric fit stability can be expressed as:

[0077] in, and These are the fitting residuals for the two planes, respectively.

[0078] (4) Confidence level of stability in dimensional calculation This is used to describe whether the size results calculated by different local sampling subsets are consistent within the current size feature region.

[0079] Suppose that sampling is obtained from the size feature region Each subset is used to calculate its size value.

[0080] but:

[0081] in, Standard deviation, This is a parameter for adjusting dimensional stability. It can be determined based on the allowable measurement error of the dimensions. Generally, it is taken as 20% to 100% of the allowable measurement error. The larger the value, the smaller the impact of dimensional fluctuations caused by repeated sampling on the confidence level.

[0082] Therefore, size confidence It includes not only image and point cloud quality, but also the geometric integrity and computational stability of the size feature itself, which better meets the needs of actual size measurement tasks.

[0083] Step 5, Low Confidence Determination ? Size confidence The value range is [0,1]. The closer the value is to 1, the higher the reliability of the measurement under the current measurement conditions; when... Lower, or even below a certain threshold If the confidence level is too low, it is considered unreliable and requires analysis of the reasons for the low confidence level and retesting. Therefore, the confidence threshold... The method for determining this is the low confidence criterion.

[0084] To make the confidence threshold Based on objective evidence, during the factory calibration, on-site calibration, or process verification stages of binocular vision measurement equipment, several calibration samples with known real dimensions are selected for measurement.

[0085] Step 5.1, for the first A calibration sample, whose true size is... The dimensions measured by the binocular vision system are The confidence level of its corresponding size is calculated according to steps 1 to 4 above. Then the measurement error corresponding to this sample for:

[0086] Through the By measuring a set of calibration samples, the following confidence and error sample sets can be obtained. This sample set is used to describe the statistical relationship between dimensional confidence level and actual measurement error. Generally, the higher the dimensional confidence level, the smaller the corresponding measurement error; the lower the dimensional confidence level, the larger the corresponding measurement error or the more significant the error fluctuation.

[0087] Step 5.2, initially determine the candidate confidence threshold. Statistically count all that satisfy The calibration sample was used, and the measurement error was calculated to be no more than the allowable measurement error. Ratio:

[0088] in, To allow for measurement error, it is typically the tolerance of the dimension being measured; This represents the proportion of measurement results that meet the error requirements when the candidate threshold is t. This indicates the number of output samples that meet the error requirement. This represents the total number of output samples that meet the candidate threshold condition. This indicates the number of elements in the set.

[0089] Step 5.3, assuming the required reliable output probability for the measurement process is... For example, it is advisable Then output threshold. The minimum candidate threshold that can be determined to satisfy the reliable output probability requirement is:

[0090] The above formula means that, among all candidate thresholds, the lowest confidence threshold is selected such that among all size measurements with a confidence level higher than this threshold, at least one... The proportions meet the allowable measurement error requirements. This method is used to determine... This can prevent low-quality dimensions from being directly output due to setting the threshold too low, and it can also prevent a large number of acceptable dimensions from being misjudged as low-confidence dimensions due to setting the threshold too high.

[0091] Determining the confidence threshold Then, the confidence level of the measured size can be determined. and The relationship is used to determine low confidence: when the confidence level of the measured dimension meets the following conditions... If the measurement meets the reliable output conditions, the measurement result can be output. Otherwise, the measurement is considered a low-confidence dimension, and the final result is not output directly. Instead, the measurement proceeds to the subsequent low-confidence cause analysis and retest guidance process.

[0092] Step 6: Quantitative calculation of the causes of low confidence.

[0093] When the size to be measured After being judged as having low confidence, further analysis was conducted to determine the main reasons for this low confidence level. The reasons for the low confidence level were not based on subjective human judgment, but rather on the size characteristic regions. The image state, matching state, and point cloud state within the image are obtained through quantitative calculation.

[0094] Define the set of reasons for low confidence as ,in, For occlusion-type low confidence, The result is a low confidence level for the brightness anomaly type. The confidence level is low for weak texture type. For low confidence due to insufficient parallax, For low confidence levels with blurred edges, For left-right matching conflict type low confidence, It represents a sparse point cloud with low confidence.

[0095] By calculating the evaluation function for each type of low-confidence cause, a set of parameters for the degree of influence of low-confidence is formed. Finally, the size to be measured The main reason for low confidence It can be calculated using the following formula:

[0096] Binocular vision measurement operators can perform relevant operations based on the prompts indicating the primary cause of low confidence.

[0097] Specifically, the evaluation functions for various low-confidence causes are as follows: (1) The essence of occlusion-related low confidence is that the size feature to be measured can be detected in one image, but cannot form a stable match in another image due to viewpoint occlusion. Therefore, the visibility of a feature can be measured by the proportion of the number of features that can be stably matched in both images to the number of detectable features in the left image. Its evaluation function can be expressed as:

[0098] in, This represents the number of detectable features within the size feature region in the left image. This represents the number of features that can be stably matched in both the left and right images. The lower this proportion, the more severe the occlusion, and the higher the confidence level for occlusion-related low-confidence features.

[0099] (2) Brightness anomalies can cause overexposure or underexposure in size feature areas, leading to abnormalities in local texture, edge grayscale, and matching cost, thus affecting binocular matching and size feature localization. The ratio of overexposed to underexposed pixels in the size feature area can effectively characterize the degree of brightness anomalies; the higher the ratio, the more likely the brightness anomaly is to be the main cause of low confidence. Therefore, the low confidence evaluation function for brightness anomalies can be expressed as:

[0100] in, For the number of overexposed pixels, The number of underexposed pixels. This represents the total number of pixels in the size feature region.

[0101] (3) Weak texture regions lack stable grayscale variations, making it easy for multiple matching or incorrect matching to occur between left and right images, leading to instability in disparity and 3D points. The grayscale variance within the size feature region can be used to represent the richness of texture; the smaller the grayscale variance, the weaker the texture. Therefore, the low-confidence evaluation function for weak texture can be expressed as:

[0102] in, The grayscale variance within the size characteristic region. These are texture adjustment parameters.

[0103] (4) Insufficient parallax usually indicates that the measured area is far from the camera, or that the current binocular baseline is too small relative to the measurement distance. Since depth error amplifies as parallax decreases, comparing the average parallax of the size feature region with the allowable lower limit of parallax, the greater the average parallax falls below the lower limit, the more likely insufficient parallax is to be the main cause of low confidence in the measured size. Therefore, the evaluation function for low confidence due to insufficient parallax can be expressed as:

[0104] in, The average disparity of the size feature region.

[0105] (5) Blurred edges can cause fluctuations in the positioning of dimensional features such as endpoints, edges, hole outlines, and step boundaries. Image gradient magnitude reflects edge sharpness; therefore, the average gradient magnitude within the dimensional feature region is used to measure edge clarity, and its evaluation function can be expressed as:

[0106] in, The average gradient magnitude within the size characteristic region. These are edge adjustment parameters.

[0107] (6) Left-right matching conflict reflects inconsistencies in the disparity results obtained for the same size feature from the left image to the right image and from the right image to the left image. This usually indicates mismatches, duplicate texture interference, or unstable occlusion boundaries in local areas. Therefore, the evaluation function for low confidence due to left-right matching conflict can be expressed as the average value of the left-right disparity difference:

[0108] in, The disparity from the left image to the right image. The disparity obtained by reverse verification from the right image to the left image. For size feature areas, This represents the total number of pixels in the size feature region.

[0109] (7) Sparse local point cloud indicates that the feature area of ​​the dimension to be measured lacks sufficient three-dimensional point support, which may lead to unstable hole contour fitting, plane fitting, or edge positioning. Therefore, the sufficiency of the point cloud is measured by the ratio of the effective number of three-dimensional points to the minimum number of points required for this type of dimension measurement:

[0110] in, The number of effective 3D points in the size feature region. The minimum number of valid 3D points required to measure this dimension.

[0111] Step 7: Generate the retesting guidance strategy.

[0112] After identifying the cause of low confidence, retesting guidance information is generated based on the low confidence cause evaluation function, dimensional feature type, dimensional feature geometric state, and current acquisition status. To generate retesting actions with clear adjustment directions and parameters for specific low-confidence dimensions, this method establishes a retesting strategy mapping function composed of "candidate action generation" and "comprehensive cost evaluation." .

[0113] For the dimension to be measured Retesting strategy mapping function It can be represented as:

[0114] in, This is a set of parameters representing the degree of influence at low confidence levels. For size characteristics, For dimensional features and geometric state, This represents the current data collection status. This is the final output of the retest guide action.

[0115] Dimensional features and geometric state Used to describe the position and orientation of the dimension to be measured in space, it can be represented as:

[0116] in, Indicates the normal direction of the local region where the dimensional feature is located. Indicates the tangential direction of the edge line, hole profile, or step boundary. Indicates the center position of the dimensional feature. This represents the average depth of the dimensional feature region. Indicates the size feature region. Current acquisition status. It can be represented as:

[0117] in, Indicates the current camera pose. Indicates the current exposure parameters. Indicates the current supplemental lighting parameters. Indicates the current focus status.

[0118] Use a single action vector to uniformly represent the retested actions:

[0119] in, Indicates the direction and distance of camera translation. Indicates the amount of adjustment of the observation angle. This indicates the amount of distance adjustment between the camera and the object being measured. Indicates the amount of exposure adjustment. Indicates the adjustment amount for the direction or intensity of the fill light. Indicates the amount of focus adjustment. This indicates the local area that was the focus of the retest.

[0120] The retesting guidance strategy generation process includes the following steps: (1) Generation of candidate retest actions Retesting strategy mapping function The first step is to generate a set of candidate retest actions based on the reasons for low confidence. The set of candidate retest actions is represented as follows:

[0121] in, For the first Rules for generating retest actions corresponding to low-confidence causes. The trigger function is defined as follows:

[0122] in, For the first Trigger thresholds for low-confidence causes.

[0123] The meaning of this formula is: when the score of a certain type of low confidence cause exceeds the corresponding threshold, the action generation rule corresponding to that cause is invoked; if multiple causes exceed the threshold at the same time, multiple candidate actions are generated and enter the subsequent cost filtering.

[0124] Specifically, when When the value is 0, it indicates low confidence due to occlusion. In this case, the size feature region is partially invisible or has a missing match in both the left and right images. The goal of the retest is to change the viewing direction so that the previously occluded region enters the binocular common visibility range. Therefore, the corresponding action generation rule... This includes changing the viewing direction, moving the camera laterally, or rotating the object being measured.

[0125] when When this value is 0, it indicates low confidence due to reflectivity or brightness anomalies. In this case, the size feature area exhibits overexposure, underexposure, or high-brightness reflection, leading to unstable edge and texture extraction. The goal of retesting is to reduce brightness anomalies and improve local imaging. Therefore, the corresponding action generation rule... This includes reducing exposure, changing the direction of fill light, or changing the viewing angle. Exposure adjustments can be expressed as:

[0126] in, This is the exposure adjustment factor. This formula indicates that the higher the score for reflections or brightness abnormalities, the greater the reduction in exposure.

[0127] when When the value is 0, it indicates a weak texture type with low confidence. At this point, the grayscale variation in the size feature region is small, making it difficult to establish a stable match between the left and right images. The goal of retesting is to improve the locally matchable texture or increase the pixel proportion of that region in the image. Therefore, the corresponding action generation rule... This includes shortening the measurement distance, enhancing auxiliary lighting, or localized magnification for data acquisition.

[0128] when When the parallax is insufficient, it indicates low confidence. In this case, the measured area is far from the camera, the parallax is too small, and depth errors are easily amplified. The goal of retesting is to bring the size feature area into a more suitable parallax range. Based on binocular vision:

[0129] If we want the average disparity to reach the target value during the retest... The target measurement distance is:

[0130] Therefore, the distance adjustment amount can be expressed as:

[0131] in, This represents the maximum allowable approach distance in a single attempt. The formula indicates the distance at the current average depth. If the parallax is too large or too small, calculate a remeasurement distance closer to the object being measured so that the remeasurement parallax falls within the range suitable for measurement.

[0132] when When the value is "0", it indicates low confidence due to edge blurring. In this case, the boundary transitions of dimensional features are unclear, and the positioning of endpoints, edges, or hole contours is unstable. The goal of retesting is to improve edge clarity. Therefore, the corresponding action generation rules... This includes refocusing, stabilizing the equipment, or partial magnification for acquisition. The focus adjustment amount can be expressed as:

[0133] in, This is the target focus position calculated based on the average depth of the size features. This formula indicates that focusing is adjusted according to the depth of the size features to make the edges of that local area sharper.

[0134] when When the value is 0, it indicates low confidence due to left-right matching conflicts. In this case, the relationship between corresponding points in the left and right images is unstable, possibly caused by repetitive textures, local reflections, occluded boundaries, or an oblique viewpoint. The goal of retesting is to change the observation conditions to make the matching relationship between the left and right images more unique. Therefore, the corresponding action generation rule... This includes changing the observation angle, shortening the measurement distance, or zooming in on specific areas for data acquisition.

[0135] when When this occurs, it indicates a sparse point cloud with low confidence. In this case, the number of effective 3D points within the size feature region is insufficient to stably complete endpoint localization, contour fitting, or plane fitting. The goal of retesting is to increase the number of effective points in this local region. Therefore, the corresponding action generation rules... This includes localized magnification acquisition, shortening the measurement distance, or changing the observation direction. The localized remeasurement area can be obtained by expanding the original-sized feature area.

[0136] in, This is the area expansion coefficient. This formula indicates that during retesting, the entire object is not re-acquired, but the acquisition range is appropriately expanded around the original low-confidence size feature area in order to obtain more valid points.

[0137] (2) Comprehensive cost evaluation Retesting strategy mapping function The second step is to conduct a comprehensive cost evaluation of each candidate retest action. In actual measurement, there may be multiple reasons for low confidence in the measured dimension, and a single low-confidence cause may require more than one. Therefore, when the set of candidate retest actions... When there are multiple candidate retest actions, the retest strategy mapping function A comprehensive cost evaluation is needed for each candidate retest action. For any candidate retest action... Its overall cost is defined as:

[0138] in, Indicates the execution of an action Risk items that still do not reach the output threshold Indicates the time cost of the action. Indicates the cost of movement or operation. Indicates the risk of action execution. These are weighting coefficients. Risk item. This can be further expressed as:

[0139] in, Indicates the execution of an action The expected size confidence level. In actual calculations, when... When the value is less than zero, it is set to zero, meaning that no penalty is applied when the expected confidence level has reached the actual output threshold.

[0140] Execute action The confidence level of the expected size can be expressed as:

[0141] in, Indicates action For the The expected improvement amount for reasons with low confidence. If the action... If it can effectively improve a certain low-confidence cause, then the corresponding Larger; if the action If it is unrelated to this reason, then the corresponding Take the smaller value or zero.

[0142] Finally, based on the above calculations, the candidate retest action with the lowest overall cost can be selected as the retest guidance action:

[0143] Therefore, the retest strategy mapping function It is concretized as a process of "cause-triggered candidate action - prediction confidence improvement - comprehensive cost screening", rather than an abstract empirical judgment function. This function can generate retesting actions with clear adjustment directions and parameters based on the cause of low confidence, size feature type, and current acquisition status, thereby achieving targeted retesting for size features.

[0144] Step 8: Retest execution and size update.

[0145] After performing the retest, the original low-confidence size feature region can be further fused with local point clouds based on the retest image results, and the size features can be recalculated. At the same time, the low confidence level is determined again: if the confidence level requirement is met, the calculated size features are output; otherwise, the quantitative analysis of the cause of low confidence, the generation of retest guidance strategy, and the calculation of size features are repeated until the confidence level requirement is met and the measurement is completed.

[0146] The binocular vision retest guidance method oriented towards size features provided in this embodiment can achieve the following beneficial effects compared with the prior art: (1) The binocular vision retesting guidance method for size features provided by the present invention takes the size features to be measured as the core evaluation object and establishes a size confidence evaluation mechanism for specific measurement features such as endpoints, edges, hole contours, fitting planes, step surfaces or local contour segments. This avoids judging the reliability of measurement results based solely on the quality of the entire image or the overall point cloud. As a result, it can more accurately identify low-confidence measurement results in local dimensions such as aperture, hole distance, edge distance, step height, and contour deviation, thereby improving the reliability of binocular vision size measurement results.

[0147] (2) The binocular vision retest guidance method for size features provided by the present invention can further determine the cause of low confidence when the size confidence is insufficient, and generate targeted retest guidance information according to different causes such as occlusion, reflection, weak texture, insufficient parallax, blurred edges, left and right matching conflict or local point cloud sparseness, thereby improving the efficiency of on-site measurement, reducing the difficulty of operation for non-professionals, and enhancing the adaptability in application scenarios such as handheld measurement, robot measurement and production line online inspection.

[0148] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0149] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0150] The above description is merely an exemplary embodiment of this disclosure and should not be construed as limiting the scope of this disclosure. Any equivalent changes and modifications made in accordance with the teachings of this disclosure shall still fall within the scope of this disclosure. Those skilled in the art will readily conceive of embodiments of this disclosure upon considering the specification and practicing the disclosure herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not described herein. The specification and embodiments are to be considered exemplary only, and the scope and spirit of this disclosure are defined by the claims.

[0151] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0152] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A binocular vision retest guidance method oriented towards size features, characterized in that, include: The left and right images of the object under test are acquired through a binocular vision system, and a three-dimensional point cloud of the object under test is generated based on the left and right images using a binocular vision algorithm. Determine at least one dimension to be measured and its corresponding dimensional feature, wherein the dimensional feature is a geometric feature used to calculate the dimension to be measured; Based on the left and right images, the left-right matching consistency, local texture effectiveness, edge sharpness, brightness effectiveness, and disparity effectiveness of each pixel in the size feature region of each size to be measured are obtained. Based on the 3D point cloud, the local stability of the point cloud of each 3D point in the size feature region of each size to be measured is obtained. Based on at least two of the following: left-right matching consistency, local texture effectiveness, edge sharpness, brightness effectiveness, disparity effectiveness, or local point cloud stability, the reliability of each point in the size feature region of each size to be measured is calculated by weighted summation. The size feature region is the set of pixels corresponding to the size features in the left and right images and the 3D points corresponding to the size features in the 3D point cloud. Calculate the size confidence level of each size based on the reliability of each point within the size characteristic region of each size to be measured; If the size confidence level does not meet the confidence level requirement, a retest is performed: For the size feature region of the size to be tested where the size confidence level does not meet the confidence level requirement, the acquisition state of the binocular vision system is adjusted, and the local left and local right images corresponding to the size feature region are reacquired. The local 3D point cloud of the size feature region is regenerated based on the local left and local right images, the reliability of each point in the size feature region is recalculated, and the size confidence level of the size to be tested is updated based on the recalculated reliability of each point until the size confidence level of the size to be tested meets the confidence level requirement. If the confidence level of the dimension meets the confidence level requirement, the measurement result of the dimension to be measured is calculated and output based on the dimension characteristics.

2. The binocular vision retest guidance method for size features as described in claim 1, characterized in that, The dimensional features include at least one of the following: the endpoints, edges, hole profiles, fitting planes, step surfaces, or local profile segments corresponding to the dimensions.

3. The binocular vision retesting guidance method oriented towards size features as described in claim 1, characterized in that, The confidence level requirement is that the size confidence level is not lower than the confidence threshold, and the method further includes: The measurement dimensions of several calibration samples with known true dimensions are measured using a binocular vision system to obtain the measured dimensions of each calibration sample. For each calibration sample, the absolute value of the difference between its true dimension and the measured dimension is calculated to obtain the measurement error corresponding to each calibration sample. The size confidence level of the measurement dimension of each calibration sample is calculated. Based on the size confidence levels and corresponding measurement errors of several calibration samples, a confidence level and error sample set is formed. Set a candidate confidence threshold, count all calibration samples in the confidence and error sample set that satisfy the size confidence level not lower than the candidate confidence threshold, and calculate the proportion of those samples whose measurement error does not exceed the allowable measurement error. Adjust the candidate confidence threshold, and determine the smallest candidate confidence threshold that satisfies the requirement that the proportion is not lower than the preset reliable output probability as the confidence threshold.

4. The binocular vision retesting guidance method oriented towards size features as described in claim 1, characterized in that, The left-right matching consistency is used to determine whether the correspondence between the left and right images is stable. The local texture validity is used to determine whether the size feature region has a matchable texture that meets the set requirements; The edge sharpness is used to determine whether the boundary or outline of the object being measured is clear; The brightness effectiveness is used to suppress overexposed, underexposed, or highly reflective areas in the left and right images; The parallax validity is used to determine whether the current parallax falls within the set parallax range; The local stability of the point cloud is used to describe the degree of fluctuation in the three-dimensional coordinates of the current point's neighborhood.

5. The binocular vision retesting guidance method oriented towards size features as described in claim 1 or 4, characterized in that, The step of calculating the size confidence level of each size to be measured based on the reliability of each point within the size characteristic region of each size to be measured includes: Calculate the average confidence level of the size feature region based on the reliability of each point within the size feature region; Based on the average confidence level of the size feature region, the size confidence level of each size to be measured is determined.

6. The binocular vision retest guidance method for size features as described in claim 5, characterized in that, The step of determining the size confidence level for each size to be measured based on the average confidence level of the size feature regions includes: The system acquires at least one of the following: geometric integrity confidence score, geometric fitting stability confidence score, and size calculation stability confidence score. The geometric integrity confidence score describes whether the key features required for the measured size are complete. The geometric fitting stability confidence score describes the fitting error of the size features. The size calculation stability confidence score describes whether the size results calculated by different local sampling subsets within the current size feature region are consistent. The size confidence of each dimension to be measured is calculated by weighting and summing at least one of the confidence in the integrity of the geometric structure, the confidence in the stability of the geometric fit, and the confidence in the stability of the size calculation, and the average confidence in the size feature region.

7. The binocular vision retesting guidance method oriented towards size features as described in claim 1, characterized in that, The step of performing a retest when the confidence level of the size does not meet the confidence level requirement includes: If the confidence level of the size does not meet the confidence level requirement, determine the set of reasons for the low confidence level; Calculate the evaluation function for each type of low-confidence cause in the set of low-confidence causes to form a set of parameters for the degree of influence of low-confidence. The type of low-confidence cause with the largest evaluation function is identified as the primary low-confidence cause. Based on the set of parameters indicating the degree of influence of low confidence, a retest is performed.

8. The binocular vision retesting guidance method oriented towards size features as described in claim 7, characterized in that, The set of low confidence reasons includes at least one of the following: occlusion-related low confidence, brightness anomaly-related low confidence, weak texture-related low confidence, insufficient parallax-related low confidence, blurred edge-related low confidence, left-right matching conflict-related low confidence, or local point cloud sparseness-related low confidence.

9. The binocular vision retesting guidance method oriented towards size features as described in claim 7, characterized in that, The step of performing a retest based on the set of low-confidence influence parameters includes: Based on the trigger function of the evaluation function of at least one type of low confidence cause in the low confidence impact parameter set and the corresponding retest action generation rule, a candidate retest action set is generated; the retest action generation rule is a set of size features, size feature geometric state and current acquisition state, the size feature geometric state is used to describe the position and orientation of the size to be measured in space, and the current acquisition state includes a set representing the current camera pose, current exposure parameters, current fill light parameters and current focus state; A comprehensive cost evaluation is performed on each candidate retest action in the candidate retest action set, and the candidate retest action with the lowest comprehensive cost is determined as the retest guidance action. Perform the retest operation according to the retest guidance instructions.

10. The binocular vision retest guidance method for size features as described in claim 8, characterized in that, The evaluation function for the occlusion-type low confidence is: in, This represents the number of features detected within the size-specific region of the left image. This represents the number of features that can be matched in both the left and right images. When the low confidence level is due to occlusion, the size feature region is partially invisible or has missing matching in the left and right images. The goal of the retest is to change the viewing direction so that the originally occluded region enters the binocular common visibility range. The corresponding retest action generation rules include at least one of changing the viewing direction, moving the camera laterally, or rotating the object being tested.

Images