Contour correction calibration method, system, device and medium based on binocular vision

Abstract

This invention relates to the field of machine vision technology, and discloses a contour correction and calibration method based on binocular vision. The method includes: scanning a trapezoidal protrusion of a ramp calibration block to obtain first and second depth maps; calculating a first boundary point and a first average height, and calculating a second boundary point and a second average height; updating the boundary points based on the average height; calculating a scale correction ratio based on the updated boundary point coordinates; correcting the coordinates of the second boundary point using the ratio, and calculating its average offset from the coordinates of the first boundary point; aligning the second depth map with the first depth map based on the offset to obtain an aligned second depth map, and calculating their height difference; scanning the object under test to obtain binocular contour data, and aligning it according to the ratio, offset, and height difference to obtain a point cloud. This invention also proposes a contour correction and calibration system, device, and storage medium based on binocular vision. This invention can improve the accuracy and efficiency of point cloud stitching in complex environments.

Description

Technical Field

[0001] This invention relates to the field of machine vision technology, and in particular to a contour correction and calibration method, system, device and medium based on binocular vision. Background Technology

[0002] When measuring workpieces with deep grooves, steep walls, or concave structures, traditional monocular sensors are prone to laser beams being blocked by the workpiece edges, creating "blind spots" that result in missing data. When detecting highly reflective objects such as metals and chips, surface specular or diffuse reflection stray light can also make point cloud extraction difficult. In contrast, binocular solutions use two cameras to capture the same laser line from different angles, enabling complementary blind spots and effectively suppressing noise and interference caused by reflections. This has made it the mainstream technology choice for high-precision industrial measurement.

[0003] The current mainstream method for binocular laser profilometer calibration involves aligning the coordinates of the point clouds captured by the two cameras through rigid or linear transformations. Before binocular calibration, each camera undergoes its own intrinsic and extrinsic parameter calibration and distortion correction. However, while this calibration method has basic application benefits, it is constrained by hardware and optical characteristics, revealing many significant shortcomings in real-world industrial scenarios. This also makes it difficult for the calibration accuracy and adaptability of binocular systems to meet the actual needs of high-precision 3D measurement. Summary of the Invention

[0004] This invention provides a contour correction and calibration method, system, device, and medium based on binocular vision, the main purpose of which is to improve the accuracy and efficiency of point cloud stitching in complex environments.

[0005] To achieve the above objectives, this invention provides a contour correction and calibration method based on binocular vision, applied to a binocular line profiler, comprising: Scan the trapezoidal protrusions on the slope calibration block to obtain the first depth map and the second depth map; The first average height of the first boundary point corresponding to the trapezoidal protrusion is calculated based on the first depth map, and the second average height of the second boundary point corresponding to the trapezoidal protrusion is calculated based on the second depth map. The first boundary point is updated based on the first average height to obtain the updated first boundary point, and the second boundary point is updated based on the second average height to obtain the updated second boundary point. Calculate the scale correction ratio based on the updated coordinates of the first boundary point and the updated coordinates of the second boundary point; The coordinates of the updated second boundary point are corrected using the scale correction ratio to obtain the corrected coordinates of the second boundary point, and the average offset between the updated first boundary point coordinates and the corrected second boundary point coordinates is calculated. The second depth map is aligned with the first depth map based on the average offset to obtain an aligned second depth map, and the height difference between the aligned second depth map and the first depth map is calculated. The preset object to be tested is scanned to obtain binocular contour data. The binocular contour data is aligned according to the scale correction ratio, the average offset and the height difference to obtain the point cloud of the object to be tested.

[0006] Optionally, calculating the first average height of the first boundary point corresponding to the trapezoidal protrusion based on the first depth map includes: Based on the first depth map, the first boundary point corresponding to the trapezoidal protrusion is obtained. The first boundary point includes multiple first left boundary points and multiple first right boundary points. Multiple first left boundary points and multiple first right boundary points are combined according to a preset scanning order to obtain multiple first boundary point pairs, wherein a first boundary point pair includes a first left boundary point and a first right boundary point. The first average height of the first boundary point corresponding to the trapezoidal protrusion is calculated based on the height of the first left boundary point and the height of the first right boundary point in each first boundary point pair.

[0007] Optionally, the first average height includes the first average height of multiple boundary point pairs. The step of calculating the first average height of the first boundary point corresponding to the trapezoidal protrusion based on the height of the first left boundary point and the height of the first right boundary point in each first boundary point pair includes: Based on the height of the first left boundary point and the height of the first right boundary point in each first boundary point pair, calculate the height of the point cloud between the first left boundary point and the first right boundary point in each first boundary point pair, wherein the point cloud between the first left boundary point and the first right boundary point in the first boundary point pair includes two endpoints, the first left boundary point and the first right boundary point. The first average height of the first left and first right boundary points in each first boundary point pair is calculated based on the height of the point cloud between the first left and first right boundary points in each first boundary point pair.

[0008] Optionally, calculating the scale correction ratio based on the updated coordinates of the first boundary point and the updated coordinates of the second boundary point includes: The distance between the left and right boundaries of the trapezoidal protrusion is calculated based on the updated coordinates of the first boundary point to obtain the width of the first trapezoidal boundary. The distance between the left and right boundaries of the trapezoidal protrusion is calculated based on the updated coordinates of the second boundary point to obtain the width of the second trapezoidal boundary. The ratio of the width of the first trapezoidal boundary to the width of the second trapezoidal boundary is calculated to obtain the scale correction ratio.

[0009] Optionally, aligning the second depth map with the first depth map based on the average offset to obtain an aligned second depth map includes: Each point cloud in the second depth map is obtained, and the average offset is used to align each point cloud in the second depth map to obtain multiple aligned point clouds; The second alignment depth map is constructed based on the multiple alignment point clouds.

[0010] Optionally, the point cloud of the object under test includes X-scale data, X-position data, and Z-axis height data. The step of aligning the binocular contour data according to the scale correction ratio, the average offset, and the height difference to obtain the point cloud of the object under test includes: An X-axis scale correction model is constructed based on the scale correction ratio; an X-axis position correction model is constructed based on the average offset; and a Z-axis height correction model is constructed based on the height difference. The binocular contour data are input into the X-axis scale correction model, the X-axis position correction model, and the Z-axis height correction model, respectively, to obtain the X-scale data, X-position data, and Z-axis height data; The binocular contour data is input into the X-axis scale correction model, and the scale of the binocular contour data is corrected according to the X-axis scale correction model to obtain the X-scale data. The binocular contour data are input into the X-axis position correction model, and the position of the binocular contour data is corrected according to the X-axis position correction model to obtain the X position data. The binocular contour data are input into the Z-axis height correction model, and the height data of the binocular contour data is corrected according to the Z-axis height correction model to obtain the Z-axis height data.

[0011] Optionally, updating the first boundary point based on the first average height to obtain the updated first boundary point includes: Based on the preset height fitting function and the first mean height, the X-axis coordinates of the first boundary point are updated to obtain the updated first boundary point. The high-fit function is:

[0012] Where a, b, c, and d are the coefficients of each term of the function, and h is the height of the first mean. This is the updated X-axis coordinate value of the first boundary point.

[0013] To address the aforementioned problems, the present invention also provides a contour correction and calibration system based on binocular vision, the system comprising: The data acquisition module is used to scan the trapezoidal protrusions on the slope calibration block to obtain the first depth map and the second depth map; The first average height of the first boundary point corresponding to the trapezoidal protrusion is calculated based on the first depth map, and the second average height of the second boundary point corresponding to the trapezoidal protrusion is calculated based on the second depth map. The scale correction module is used to update the first boundary point according to the first average height to obtain the updated first boundary point, and to update the second boundary point according to the second average height to obtain the updated second boundary point; Calculate the scale correction ratio based on the updated coordinates of the first boundary point and the updated coordinates of the second boundary point; The position correction module is used to correct the coordinates of the updated second boundary point using the scale correction ratio, to obtain the corrected coordinates of the second boundary point, and to calculate the average offset between the updated coordinates of the first boundary point and the corrected coordinates of the second boundary point. The point cloud alignment module is used to align the second depth map with the first depth map according to the average offset to obtain an aligned second depth map, and to calculate the height difference between the aligned second depth map and the first depth map. The preset object to be tested is scanned to obtain binocular contour data. The binocular contour data is aligned according to the scale correction ratio, the average offset and the height difference to obtain the point cloud of the object to be tested.

[0014] To address the above problems, the present invention also provides an electronic device, the electronic device comprising: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the binocular vision-based contour correction calibration method as described above.

[0015] To address the aforementioned problems, the present invention also provides a computer-readable storage medium, including a data storage area and a program storage area. The data storage area stores created data, and the program storage area stores a computer program. When the computer program is executed by a processor, it implements the binocular vision-based contour correction calibration method as described above.

[0016] In this embodiment of the invention, the first and second depth maps are obtained by scanning the trapezoidal protrusion of the ramp calibration block; the first boundary point and the first average height are calculated, and the second boundary point and the second average height are calculated; the boundary points are updated according to the average height; the scale correction ratio is calculated according to the updated boundary point coordinates; the coordinates of the second boundary point are corrected using the ratio, and the average offset between the second boundary point coordinates and the first boundary point coordinates is calculated; the second depth map and the first depth map are aligned according to the offset to obtain the aligned second depth map, and the height difference is calculated; the object under test is scanned to obtain binocular contour data, and the point cloud is obtained by aligning according to the ratio, offset, and height difference. Therefore, the contour correction calibration method, system, electronic device, and computer-readable storage medium based on binocular vision proposed in this invention, through the design of the ramp trapezoidal calibration block and the step function fitting strategy, enables the system to quickly and fully automatically complete multi-dimensional alignment of X scale, X position, and Z height without relying on high-precision external measuring equipment and manual intervention. This solves the problems of point cloud misalignment and layering caused by non-orthogonal coordinate systems and temperature drift, as well as the difficulty of production line calibration, and improves the accuracy and calibration efficiency of point cloud stitching in complex environments. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating a contour correction and calibration method based on binocular vision, provided in an embodiment of the present invention. Figure 2 This is a schematic diagram of a contour correction and calibration system based on binocular vision provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the internal structure of an electronic device that implements a contour correction and calibration method based on binocular vision, according to an embodiment of the present invention.

[0018] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0019] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0020] This application provides a contour correction and calibration method based on binocular vision. The execution entity of the binocular vision-based contour correction and calibration method includes, but is not limited to, at least one of the following electronic devices that can be configured to execute the method provided in this application: a server, a terminal, etc. The server can be a standalone server or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. In other words, the binocular vision-based contour correction and calibration method can be executed by software or hardware installed on a remote device or server-side device, and the software can be a blockchain platform. The server includes, but is not limited to, a single server, a server cluster, a cloud server, or a cloud server cluster.

[0021] Reference Figure 1 The diagram shown is a flowchart illustrating a contour correction and calibration method based on binocular vision according to an embodiment of the present invention. In this embodiment, the contour correction and calibration method based on binocular vision applied to a binocular contour meter includes the following steps S1-S7: S1. Scan the trapezoidal protrusion on the slope calibration block to obtain the first depth map and the second depth map.

[0022] Understandably, scanning the slope calibration block with a binocular profilometer can acquire continuous contour data covering the entire height range in one go, and obtain depth maps from the left and right cameras respectively. The trapezoidal protrusions on the calibration block provide a stable geometric reference for subsequent extraction of boundary features.

[0023] Among them, the binocular line profilometer refers to a measurement system consisting of two cameras (left and right) and a line laser. During operation, the line laser projects a laser line onto the surface of the object, and the two cameras capture images of the laser line from different angles. Based on the principle of laser triangulation, each camera calculates the three-dimensional coordinates (i.e., profile data) of each point on the laser line, thereby achieving non-contact measurement of the surface morphology of the object.

[0024] Furthermore, when scanning the trapezoidal protrusions on the slope calibration block, the scanning will be performed according to the slope direction, specifically the inclination direction of the slope surface on the calibration block, that is, the movement direction of the profilometer during scanning is from the bottom edge of the slope to the top edge of the slope.

[0025] In this embodiment, the external shaft drives the profilometer to move in a direction parallel to the slope surface to ensure that the vertical distance between the camera and the slope remains unchanged during the scanning process, while the height (measured value) of the laser line irradiation point changes continuously with the slope position, thereby obtaining different height profile data covering the entire range.

[0026] The first and second depth maps refer to the ordered point cloud data generated after the left and right cameras scan the calibration block, respectively, and are stored in the form of a two-dimensional matrix. The row numbers of the matrix correspond to the scanning order (i.e., position in the Y direction), the column numbers correspond to the lateral positions on the laser lines (i.e., positions in the X direction), and the value of each matrix element represents the height (Z coordinate) of that point. The depth maps are the foundational data for subsequent calibration algorithms.

[0027] The trapezoidal protrusion refers to a trapezoidal structure designed on the slope of the calibration block. This structure creates a significant gradient change in the depth map, facilitating the algorithm's accurate extraction of the left and right boundary points. By analyzing the differences in the imaging position of the trapezoidal boundary at different heights, a nonlinear mapping relationship between the left and right camera coordinate systems can be established.

[0028] S2. Calculate the first average height of the first boundary point corresponding to the trapezoidal protrusion based on the first depth map, and calculate the second average height of the second boundary point corresponding to the trapezoidal protrusion based on the second depth map.

[0029] Understandably, by calculating the average height of the boundary points of the trapezoidal protrusions on the two depth maps respectively, the physical characteristics of the calibration block can be transformed into the average height of the left and right cameras respectively, providing basic data for establishing the coordinate mapping relationship between the two cameras.

[0030] The first average height refers to the average height calculated from the first depth map (such as the left camera) of the top region of the trapezoidal protrusions in each row of contours (i.e., the pixel segment between the left and right boundary points). It reflects the average height value of the top of the trapezoid in that row as measured by the left camera, and is used to establish a functional relationship with the boundary point positions later.

[0031] Similarly, the second mean height refers to the average height calculated from the top region of the trapezoidal protrusions of each row of contours in the second depth map (e.g., the right camera). It serves as an independent variable for subsequent steps such as scale correction and offset fitting. Both the first mean height and the second mean height are statistical values ​​of the height of the top region of the trapezoids in their respective depth maps, representing the height level of that row of contours, and they vary with the scanning position.

[0032] Furthermore, before calculating the first average height of the first boundary point corresponding to the trapezoidal protrusion based on the first depth map, the method further includes constructing a calibration coordinate system based on the first depth and the second depth.

[0033] In this calibration coordinate system, the X-axis is parallel to the ramp calibration block and perpendicular to the scanning motion direction of the binocular profilometer; the Y-axis is parallel to the scanning motion direction of the binocular profilometer; and the Z-axis is the cross product of the X and Y axes. The origin of the coordinate system is located at (col / 2, 0, 0), where col is the image width, col / 2 indicates that the origin of the X-axis is located in the center column of the depth map, the first 0 indicates that the origin of the Y-axis is located in the first row of the depth map (i.e., the scanning start position), and the second 0 indicates that the origin of the Z-axis is located in the reference plane (usually a virtual plane at the intersection of the camera optical axis and the laser plane).

[0034] Further, calculating the first average height of the first boundary point corresponding to the trapezoidal protrusion based on the first depth map includes: Based on the first depth map, the first boundary point corresponding to the trapezoidal protrusion is obtained. The first boundary point includes multiple first left boundary points and multiple first right boundary points. Multiple first left boundary points and multiple first right boundary points are combined according to a preset scanning order to obtain multiple first boundary point pairs, wherein a first boundary point pair includes a first left boundary point and a first right boundary point. The first average height of the first boundary point corresponding to the trapezoidal protrusion is calculated based on the height of the first left boundary point and the height of the first right boundary point in each first boundary point pair.

[0035] Furthermore, the first depth map is analyzed row by row. The gradient of adjacent pixels in each row is calculated. The column position where the gradient maximum value is located is taken as the first left boundary point, and the column position where the gradient minimum value is located is taken as the first right boundary point, thereby extracting the left and right edge positions of the trapezoidal protrusion in each row.

[0036] The first boundary point pair refers to a pair of boundary points formed by combining a first left boundary point and a first right boundary point in the same row of contours according to the scanning order, which is used to characterize the left and right edge positions of the trapezoidal protrusion on that row.

[0037] The height of the first left boundary point refers to the measured height value at the location of the first left boundary point (i.e., the left edge of the trapezoid); the height of the first right boundary point refers to the measured height value at the location of the first right boundary point (i.e., the right edge of the trapezoid).

[0038] The first mean height includes the first mean height of multiple boundary point pairs.

[0039] Further, the step of calculating the first average height of the first boundary point corresponding to the trapezoidal protrusion based on the height of the first left boundary point and the height of the first right boundary point in each first boundary point pair includes: Based on the height of the first left boundary point and the height of the first right boundary point in each first boundary point pair, calculate the height of the point cloud between the first left boundary point and the first right boundary point in each first boundary point pair, wherein the point cloud between the first left boundary point and the first right boundary point in the first boundary point pair includes two endpoints, the first left boundary point and the first right boundary point. The first average height of the first left and first right boundary points in each first boundary point pair is calculated based on the height of the point cloud between the first left and first right boundary points in each first boundary point pair.

[0040] Furthermore, the first mean height, which includes the first mean height of multiple boundary point pairs, is used to illustrate that the first mean height is not a single value, but a set of mean heights corresponding to multiple boundary point pairs (i.e., multiple lines of contours), and has its sequence properties. This provides an explanation of the data organization method for subsequent function fitting using the mean data at these different heights.

[0041] S3. Update the first boundary point according to the first average height to obtain the updated first boundary point, and update the second boundary point according to the second average height to obtain the updated second boundary point.

[0042] Understandably, by updating the first boundary point based on the first average height and the second boundary point based on the second average height, the original pixel-level boundary position can be improved to sub-pixel accuracy, thereby significantly improving the accuracy of trapezoidal edge localization, providing more accurate feature points for subsequent scale correction and position alignment, and reducing point cloud stitching errors.

[0043] The updated first boundary point refers to the sub-pixel precision boundary point recalculated based on a cubic function fit between the first average height (the average height of the top region of each row of trapezoids) calculated from the first depth map and the original pixel-level boundary points (left and right boundary points). It has higher positioning accuracy than the original boundary points and can reduce subsequent stitching errors.

[0044] Similarly, the effect of the updated second boundary point is the same, so it will not be elaborated here.

[0045] Further, updating the first boundary point based on the first average height to obtain the updated first boundary point includes: Based on the preset height fitting function and the first mean height, the X-axis coordinates of the first boundary point are updated to obtain the updated first boundary point. The high-fit function is: , Where a, b, c, and d are the coefficients of each term of the function, and h is the height of the first mean. This is the updated X-axis coordinate value of the first boundary point.

[0046] The height fitting function is a cubic polynomial function with the first mean height as the independent variable and the X-axis coordinate of the first boundary point as the dependent variable. It is used to improve the pixel-level boundary points to sub-pixel precision updated boundary points.

[0047] Furthermore, the updated first boundary point is divided into a first left boundary point and a first right boundary point. Since the X-axis coordinates of both the first left boundary point and the first right boundary point satisfy a cubic polynomial function model, the formula for calculating the X-axis coordinate value of the first right boundary point is: The formula for calculating the X-axis coordinate of the first left boundary point is: .

[0048] S4. Calculate the scale correction ratio based on the updated coordinates of the first boundary point and the updated coordinates of the second boundary point.

[0049] Understandably, by calculating the scale correction ratio based on the updated coordinates of the first and second boundary points, the physical scale difference between the left and right cameras in the X direction can be quantified, and the point cloud stitching misalignment caused by scale inconsistency can be eliminated through subsequent scaling correction, laying the foundation for high-precision 3D reconstruction.

[0050] The scale correction ratio refers to the ratio of the width of the trapezoidal protrusion in the first depth map to the width of the trapezoidal protrusion in the second depth map. It is used to eliminate the physical scale difference between the left and right cameras in the X direction by scaling the second depth map to align it with the scale of the first depth map.

[0051] The updated coordinates of the first boundary point refer to the X-axis coordinates of the left and right boundary points with sub-pixel precision, which are obtained by fitting the original pixel-level boundary points in the first depth map with the corresponding mean height using a cubic function. The updated coordinates of the second boundary point refer to the X-axis coordinates of the left and right boundary points with sub-pixel precision, which are obtained by fitting the original pixel-level boundary points in the second depth map with the corresponding mean height using a cubic function.

[0052] Further, the step of calculating the scale correction ratio based on the updated coordinates of the first boundary point and the updated coordinates of the second boundary point includes: The distance between the left and right boundaries of the trapezoidal protrusion is calculated based on the updated coordinates of the first boundary point to obtain the width of the first trapezoidal boundary. The distance between the left and right boundaries of the trapezoidal protrusion is calculated based on the updated coordinates of the second boundary point to obtain the width of the second trapezoidal boundary. The ratio of the width of the first trapezoidal boundary to the width of the second trapezoidal boundary is calculated to obtain the scale correction ratio.

[0053] The width of the first trapezoidal boundary refers to the difference calculated based on the X-axis coordinates of the left and right boundary points in the updated first boundary points, which is the physical width of the trapezoidal protrusion in the horizontal direction in the first depth map.

[0054] The second trapezoidal boundary width refers to the difference calculated based on the X-axis coordinates of the left and right boundary points in the updated second boundary points, which is the physical width of the trapezoidal protrusion in the horizontal direction in the second depth map.

[0055] The width ratio refers to the ratio of the width of the first trapezoidal boundary to the width of the second trapezoidal boundary. It is used to quantify the physical scale difference between the left and right cameras in the X direction, and serves as a scaling factor for subsequent scaling correction of the second depth map.

[0056] S5. Correct the coordinates of the updated second boundary point using the scale correction ratio to obtain the corrected coordinates of the second boundary point, and calculate the average offset between the updated first boundary point coordinates and the corrected second boundary point coordinates.

[0057] Understandably, the boundary points of the second depth map are scaled and corrected using a scale correction ratio (correcting the second boundary points). Then, the positional offset between the corrected second boundary points and the first boundary points is calculated, and the average of the left and right boundary offsets is taken to provide translation for positional alignment in the X direction.

[0058] The correction of the second boundary point's coordinates refers to the result of scaling the updated second boundary point's (sub-pixel precision) X-coordinate using a scale correction ratio, so that the width of the trapezoidal protrusion in the second depth map is equal to that in the first depth map. Figure 1 To.

[0059] The average offset refers to the average of the offset obtained by subtracting the left and right boundary coordinates of the corrected second boundary point from the left and right boundary coordinates of the first boundary point. It is used to characterize the distance that the second depth map needs to be translated as a whole in the X direction in order to achieve alignment with the position of the first depth map.

[0060] In this embodiment of the invention, when calculating the average offset between the updated coordinates of the first boundary point and the corrected coordinates of the second boundary point, it is necessary to calculate the offset between the left boundary coordinates of the first boundary point and the left boundary coordinates of the second boundary point. Then, this offset is compared with the offset between the right boundary coordinates of the first boundary point and the boundary coordinates of the second boundary point. If the difference between the two is greater than the pixel resolution of the first depth map, the calibration board is placed incorrectly, and the calibration block needs to be repositioned. Then, the first depth map and the second depth map are rescanned and obtained. The pixel resolution of the first depth map and the pixel resolution of the second depth map are the same.

[0061] S6. Align the second depth map with the first depth map according to the average offset to obtain an aligned second depth map, and calculate the height difference between the aligned second depth map and the first depth map.

[0062] Understandably, the second depth map is shifted as a whole based on the average offset to align it with the first depth map in the X direction (resulting in an aligned second depth map). Then, the height difference between the two is calculated pixel by pixel to obtain a correction model in the Z direction that varies with the height difference of each column and the current height.

[0063] The alignment of the second depth map is a depth map aligned in the X direction. It means that the depth map is obtained by translating each row of the second depth map in the X direction according to the average offset, so that the second depth map is aligned with the first depth map in the X direction.

[0064] The height difference can be expressed as the height difference per column, which refers to the difference between the measured height values ​​of the second depth map and the first depth map at the same pixel position (same row and same column). It is used to construct the height correction model of each column in the Z direction to eliminate the deviation of the two cameras in the height direction.

[0065] Further, aligning the second depth map with the first depth map based on the average offset to obtain an aligned second depth map includes: Each point cloud in the second depth map is obtained, and the average offset is used to align each point cloud in the second depth map to obtain multiple aligned point clouds; The second alignment depth map is constructed based on the multiple alignment point clouds.

[0066] Aligning point clouds refers to the process of translating and correcting the X coordinates of each original point cloud in the second depth map according to the average offset to obtain new point cloud coordinates, so that the corrected point cloud is aligned with the point cloud at the corresponding position in the first depth map in the X direction.

[0067] S7. Scan the preset object to be tested to obtain binocular contour data. Align the binocular contour data according to the scale correction ratio, the average offset and the height difference to obtain the point cloud of the object to be tested.

[0068] Understandably, when actually measuring the object to be measured, the scanned binocular contour data is transformed frame by frame by applying scale correction ratio, average offset and height difference to make the contours of the two cameras accurately aligned and finally fused into a complete 3D point cloud without misalignment or layering.

[0069] Among them, the preset object to be measured refers to the actual target workpiece that needs to be measured or inspected in three dimensions after the binocular profilometer has been calibrated, such as the mobile phone frame, blade battery and other specific equipment.

[0070] Among them, binocular contour data refers to the contour sequence (i.e., ordered point cloud data) obtained by the left and right cameras respectively after scanning the preset object to be measured using a binocular line laser profilometer.

[0071] The point cloud of the object under test includes X-scale data, X-position data, and Z-axis height data. It refers to the complete and misaligned 3D point cloud obtained by sequentially applying scale correction ratio, average offset, and height difference of each column to the binocular contour data for alignment and fusion.

[0072] Further, aligning the binocular contour data according to the scale correction ratio, the average offset, and the height difference to obtain the point cloud of the object under test includes: An X-axis scale correction model is constructed based on the scale correction ratio; an X-axis position correction model is constructed based on the average offset; and a Z-axis height correction model is constructed based on the height difference. The binocular contour data are input into the X-axis scale correction model, the X-axis position correction model, and the Z-axis height correction model, respectively, to obtain the X-scale data, X-position data, and Z-axis height data; The binocular contour data is input into the X-axis scale correction model, and the scale of the binocular contour data is corrected according to the X-axis scale correction model to obtain the X-scale data. The binocular contour data are input into the X-axis position correction model, and the position of the binocular contour data is corrected according to the X-axis position correction model to obtain the X position data. The binocular contour data are input into the Z-axis height correction model, and the height data of the binocular contour data is corrected according to the Z-axis height correction model to obtain the Z-axis height data.

[0073] The X-axis scale correction model refers to a mathematical model established based on the scale correction ratio (a quadratic function related to height), which is used to scale and correct the contour data at any height in the second depth map (or the right camera) in the X direction.

[0074] The X-axis position correction model refers to a mathematical model established based on the average offset (a quintic function related to height), which is used to perform X-direction translation correction on contour data at any height in the second depth map.

[0075] Among them, the Z-axis height correction model refers to a mathematical model (one model per column) established based on the height difference and mean height of any column in the second depth map (a set of quintic functions related to the height difference and mean of each column), which is used to correct the Z-direction offset of the height value of any pixel position in the second depth map.

[0076] Among them, X-scale data refers to the new X-coordinate data obtained by scaling the right camera contour data in the binocular contour data according to the X-axis scale correction model; X-position data refers to the X-coordinate data obtained by translating the right camera contour data in the binocular contour data according to the X-axis position correction model; and Z-axis height data refers to the Z-coordinate data obtained by correcting the height value of the right camera contour data in the binocular contour data according to the Z-axis height correction model.

[0077] In this embodiment of the invention, the first and second depth maps are obtained by scanning the trapezoidal protrusion of the ramp calibration block; the first boundary point and the first average height are calculated, and the second boundary point and the second average height are calculated; the boundary points are updated according to the average height; the scale correction ratio is calculated according to the updated boundary point coordinates; the coordinates of the second boundary point are corrected using the ratio, and the average offset between the second boundary point coordinates and the first boundary point coordinates is calculated; the second depth map and the first depth map are aligned according to the offset to obtain the aligned second depth map, and the height difference is calculated; the object under test is scanned to obtain binocular contour data, and the point cloud is obtained by aligning according to the ratio, offset, and height difference. Therefore, the contour correction calibration method, system, electronic device, and computer-readable storage medium based on binocular vision proposed in this invention, through the design of the ramp trapezoidal calibration block and the step function fitting strategy, enables the system to quickly and fully automatically complete multi-dimensional alignment of X scale, X position, and Z height without relying on high-precision external measuring equipment and manual intervention. This solves the problems of point cloud misalignment and layering caused by non-orthogonal coordinate systems and temperature drift, as well as the difficulty of production line calibration, and improves the accuracy and calibration efficiency of point cloud stitching in complex environments.

[0078] like Figure 2 The diagram shown is a schematic diagram of the contour correction and calibration system based on binocular vision according to the present invention.

[0079] The binocular vision-based contour correction and calibration system 100 described in this invention can be installed in an electronic device. Depending on the functions implemented, the binocular vision-based contour correction and calibration system may include a data acquisition module 101, a scale correction module 102, a position correction module 103, and a point cloud alignment module 104. The module described in this invention can also be referred to as a unit, which refers to a series of computer program segments that can be executed by the processor of an electronic device and can perform a fixed function, and are stored in the memory of the electronic device.

[0080] In this embodiment, the functions of each module / unit are as follows: The data acquisition module 101 is used to scan the trapezoidal protrusion on the slope calibration block to obtain a first depth map and a second depth map; The first average height of the first boundary point corresponding to the trapezoidal protrusion is calculated based on the first depth map, and the second average height of the second boundary point corresponding to the trapezoidal protrusion is calculated based on the second depth map. The scale correction module 102 is used to update the first boundary point according to the first average height to obtain the updated first boundary point, and to update the second boundary point according to the second average height to obtain the updated second boundary point. Calculate the scale correction ratio based on the updated coordinates of the first boundary point and the updated coordinates of the second boundary point; The position correction module 103 is used to correct the coordinates of the updated second boundary point using the scale correction ratio, to obtain the corrected coordinates of the second boundary point, and to calculate the average offset between the updated coordinates of the first boundary point and the corrected coordinates of the second boundary point. The point cloud alignment module 104 is used to align the second depth map with the first depth map according to the average offset to obtain an aligned second depth map, and to calculate the height difference between the aligned second depth map and the first depth map. The preset object to be tested is scanned to obtain binocular contour data. The binocular contour data is aligned according to the scale correction ratio, the average offset and the height difference to obtain the point cloud of the object to be tested.

[0081] In detail, the modules in the binocular vision-based contour correction and calibration system 100 described in this embodiment of the invention employ the same methods as described above. Figure 1 The method used is the same as the binocular vision-based contour correction and calibration method and can produce the same technical effect, so it will not be described in detail here.

[0082] like Figure 3 The diagram shown is a structural schematic of an electronic device that implements the contour correction and calibration method based on binocular vision according to the present invention.

[0083] The electronic device may include a processor 10, a memory 11, a communication bus 12 and a communication interface 13, and may also include a computer program stored in the memory 11 and executable on the processor 10, such as a contour correction calibration program based on binocular vision.

[0084] In some embodiments, the processor 10 may be composed of integrated circuits, such as a single packaged integrated circuit or multiple integrated circuits with the same or different functions, including combinations of one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and various control chips. The processor 10 is the control unit of the electronic device, connecting various components of the entire electronic device through various interfaces and lines. It executes programs or modules stored in the memory 11 (e.g., executing a contour correction calibration program based on binocular vision) and calls data stored in the memory 11 to perform various functions of the electronic device and process data.

[0085] The memory 11 includes at least one type of readable storage medium, including flash memory, portable hard drive, multimedia card, card-type memory (e.g., SD or DX memory), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 11 can be an internal storage unit of an electronic device, such as a portable hard drive. In other embodiments, the memory 11 can be an external storage device of the electronic device, such as a plug-in portable hard drive, Smart Media Card (SMC), Secure Digital (SD) card, Flash Card, etc. Furthermore, the memory 11 can include both internal and external storage units of the electronic device. The memory 11 can be used not only to store application software and various types of data installed on the electronic device, such as code for a contour correction calibration program based on binocular vision, but also to temporarily store data that has been output or will be output.

[0086] The communication bus 12 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This bus can be divided into an address bus, a data bus, a control bus, etc. The bus is configured to enable communication between the memory 11 and at least one processor 10, etc.

[0087] The communication interface 13 is used for communication between the aforementioned electronic device and other devices, including a network interface and a user interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a Wi-Fi interface, Bluetooth interface, etc.), typically used to establish communication connections between the electronic device and other electronic devices. The user interface may be a display, an input unit (such as a keyboard), or optionally, a standard wired or wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen, etc. The display may also be appropriately referred to as a screen or display unit, used to display information processed in the electronic device and to display a visual user interface.

[0088] Figure 3 Only electronic devices with components are shown; it will be understood by those skilled in the art that... Figure 3 The structure shown does not constitute a limitation on the electronic device and may include fewer or more components than shown, or combine certain components, or have different component arrangements.

[0089] For example, although not shown, the electronic device may also include a power supply (such as a battery) to power the various components. Preferably, the power supply can be logically connected to the at least one processor 10 through a power management device, thereby enabling functions such as charging management, discharging management, and power consumption management. The power supply may also include one or more DC or AC power supplies, recharging devices, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components. The electronic device may also include various sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be described in detail here.

[0090] It should be understood that the embodiments described are for illustrative purposes only and are not limited to this structure in the scope of the patent application.

[0091] The contour correction calibration program based on binocular vision stored in the memory 11 of the electronic device is a combination of multiple computer programs. When run in the processor 10, it can achieve the following: Scan the trapezoidal protrusions on the slope calibration block to obtain the first depth map and the second depth map; The first average height of the first boundary point corresponding to the trapezoidal protrusion is calculated based on the first depth map, and the second average height of the second boundary point corresponding to the trapezoidal protrusion is calculated based on the second depth map. The first boundary point is updated based on the first average height to obtain the updated first boundary point, and the second boundary point is updated based on the second average height to obtain the updated second boundary point. Calculate the scale correction ratio based on the updated coordinates of the first boundary point and the updated coordinates of the second boundary point; The coordinates of the updated second boundary point are corrected using the scale correction ratio to obtain the corrected coordinates of the second boundary point, and the average offset between the updated first boundary point coordinates and the corrected second boundary point coordinates is calculated. The second depth map is aligned with the first depth map based on the average offset to obtain an aligned second depth map, and the height difference between the aligned second depth map and the first depth map is calculated. The preset object to be tested is scanned to obtain binocular contour data. The binocular contour data is aligned according to the scale correction ratio, the average offset and the height difference to obtain the point cloud of the object to be tested.

[0092] Specifically, the processor 10's implementation method of the above-mentioned computer program can be found in [reference needed]. Figure 1 The descriptions of the relevant steps in the corresponding embodiments are not repeated here.

[0093] Furthermore, if the modules / units integrated into the electronic device are implemented as software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium. The computer-readable storage medium can be volatile or non-volatile. For example, the computer-readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, or a read-only memory (ROM).

[0094] The present invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor of an electronic device, can perform the following: Scan the trapezoidal protrusions on the slope calibration block to obtain the first depth map and the second depth map; The first average height of the first boundary point corresponding to the trapezoidal protrusion is calculated based on the first depth map, and the second average height of the second boundary point corresponding to the trapezoidal protrusion is calculated based on the second depth map. The first boundary point is updated based on the first average height to obtain the updated first boundary point, and the second boundary point is updated based on the second average height to obtain the updated second boundary point. Calculate the scale correction ratio based on the updated coordinates of the first boundary point and the updated coordinates of the second boundary point; The coordinates of the updated second boundary point are corrected using the scale correction ratio to obtain the corrected coordinates of the second boundary point, and the average offset between the updated first boundary point coordinates and the corrected second boundary point coordinates is calculated. The second depth map is aligned with the first depth map based on the average offset to obtain an aligned second depth map, and the height difference between the aligned second depth map and the first depth map is calculated. The preset object to be tested is scanned to obtain binocular contour data. The binocular contour data is aligned according to the scale correction ratio, the average offset and the height difference to obtain the point cloud of the object to be tested.

[0095] In the several embodiments provided by this invention, it should be understood that the disclosed devices, systems, and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and other division methods may be used in actual implementation.

[0096] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0097] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional modules.

[0098] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0099] Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be embraced within the invention. No appended diagram markings in the claims should be construed as limiting the scope of the claims.

[0100] The blockchain referred to in this invention is a novel application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms. Essentially, a blockchain is a decentralized database, a chain of data blocks linked together using cryptographic methods. Each data block contains information about a batch of network transactions, used to verify the validity of the information (anti-counterfeiting) and generate the next block. A blockchain can include an underlying blockchain platform, a platform product service layer, and an application service layer.

[0101] The embodiments of this application can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence (AI) refers to the theories, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0102] Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or systems stated in a system claim may also be implemented by a single unit or system through software or hardware. The term "second class" is used to indicate names and does not indicate any specific order.

[0103] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A contour correction and calibration method based on binocular vision, characterized in that, Applied to a binocular profilometer, the method includes: Scan the trapezoidal protrusions on the slope calibration block to obtain the first depth map and the second depth map; The first average height of the first boundary point corresponding to the trapezoidal protrusion is calculated based on the first depth map, and the second average height of the second boundary point corresponding to the trapezoidal protrusion is calculated based on the second depth map. The first boundary point is updated based on the first average height to obtain the updated first boundary point, and the second boundary point is updated based on the second average height to obtain the updated second boundary point. Calculate the scale correction ratio based on the updated coordinates of the first boundary point and the updated coordinates of the second boundary point; The coordinates of the updated second boundary point are corrected using the scale correction ratio to obtain the corrected coordinates of the second boundary point, and the average offset between the updated first boundary point coordinates and the corrected second boundary point coordinates is calculated. The second depth map is aligned with the first depth map based on the average offset to obtain an aligned second depth map, and the height difference between the aligned second depth map and the first depth map is calculated. The preset object to be tested is scanned to obtain binocular contour data. The binocular contour data is aligned according to the scale correction ratio, the average offset and the height difference to obtain the point cloud of the object to be tested. The step of updating the first boundary point based on the first average height to obtain the updated first boundary point includes: Based on the preset height fitting function and the first mean height, the X-axis coordinates of the first boundary point are updated to obtain the updated first boundary point. The high-fit function is: ； Where a, b, c, and d are the coefficients of each term of the function, and h is the height of the first mean. The updated X-axis coordinates of the first boundary point; The step of calculating the scale correction ratio based on the updated coordinates of the first boundary point and the updated coordinates of the second boundary point includes: The distance between the left and right boundaries of the trapezoidal protrusion is calculated based on the updated coordinates of the first boundary point to obtain the width of the first trapezoidal boundary. The distance between the left and right boundaries of the trapezoidal protrusion is calculated based on the updated coordinates of the second boundary point to obtain the width of the second trapezoidal boundary. Calculate the width ratio of the first trapezoidal boundary width to the second trapezoidal boundary width to obtain the scale correction ratio; The point cloud of the object under test includes X-scale data, X-position data, and Z-axis height data. The step of aligning the binocular contour data according to the scale correction ratio, the average offset, and the height difference to obtain the point cloud of the object under test includes: An X-axis scale correction model is constructed based on the scale correction ratio; an X-axis position correction model is constructed based on the average offset; and a Z-axis height correction model is constructed based on the height difference. The binocular contour data are input into the X-axis scale correction model, the X-axis position correction model, and the Z-axis height correction model, respectively, to obtain the X-scale data, X-position data, and Z-axis height data; The binocular contour data is input into the X-axis scale correction model, and the scale of the binocular contour data is corrected according to the X-axis scale correction model to obtain the X-scale data. The binocular contour data are input into the X-axis position correction model, and the position of the binocular contour data is corrected according to the X-axis position correction model to obtain the X position data. The binocular contour data are input into the Z-axis height correction model, and the height data of the binocular contour data is corrected according to the Z-axis height correction model to obtain the Z-axis height data.

2. The contour correction and calibration method based on binocular vision as described in claim 1, characterized in that, The step of calculating the first average height of the first boundary point corresponding to the trapezoidal protrusion based on the first depth map includes: Based on the first depth map, the first boundary point corresponding to the trapezoidal protrusion is obtained. The first boundary point includes multiple first left boundary points and multiple first right boundary points. Multiple first left boundary points and multiple first right boundary points are combined according to a preset scanning order to obtain multiple first boundary point pairs, wherein a first boundary point pair includes a first left boundary point and a first right boundary point. The first average height of the first boundary point corresponding to the trapezoidal protrusion is calculated based on the height of the first left boundary point and the height of the first right boundary point in each first boundary point pair.

3. The contour correction and calibration method based on binocular vision as described in claim 2, characterized in that, The first average height includes the first average height of multiple boundary point pairs. The step of calculating the first average height of the first boundary point corresponding to the trapezoidal protrusion based on the height of the first left boundary point and the height of the first right boundary point in each first boundary point pair includes: Based on the height of the first left boundary point and the height of the first right boundary point in each first boundary point pair, calculate the height of the point cloud between the first left boundary point and the first right boundary point in each first boundary point pair, wherein the point cloud between the first left boundary point and the first right boundary point in the first boundary point pair includes two endpoints, the first left boundary point and the first right boundary point. The first average height of the first left and first right boundary points in each first boundary point pair is calculated based on the height of the point cloud between the first left and first right boundary points in each first boundary point pair.

4. The contour correction and calibration method based on binocular vision as described in claim 1, characterized in that, The step of aligning the second depth map with the first depth map based on the average offset to obtain an aligned second depth map includes: Each point cloud in the second depth map is obtained, and the average offset is used to align each point cloud in the second depth map to obtain multiple aligned point clouds; The second alignment depth map is constructed based on the multiple alignment point clouds.

5. A contour correction and calibration system based on binocular vision, characterized in that, The system includes: The data acquisition module is used to scan the trapezoidal protrusions on the slope calibration block to obtain the first depth map and the second depth map; The first average height of the first boundary point corresponding to the trapezoidal protrusion is calculated based on the first depth map, and the second average height of the second boundary point corresponding to the trapezoidal protrusion is calculated based on the second depth map. The scale correction module is used to update the first boundary point according to the first average height to obtain the updated first boundary point, and to update the second boundary point according to the second average height to obtain the updated second boundary point; Calculate the scale correction ratio based on the updated coordinates of the first boundary point and the updated coordinates of the second boundary point; The position correction module is used to correct the coordinates of the updated second boundary point using the scale correction ratio, to obtain the corrected coordinates of the second boundary point, and to calculate the average offset between the updated coordinates of the first boundary point and the corrected coordinates of the second boundary point. The point cloud alignment module is used to align the second depth map with the first depth map according to the average offset to obtain an aligned second depth map, and to calculate the height difference between the aligned second depth map and the first depth map. The preset object to be tested is scanned to obtain binocular contour data. The binocular contour data is aligned according to the scale correction ratio, the average offset and the height difference to obtain the point cloud of the object to be tested. The step of updating the first boundary point based on the first average height to obtain the updated first boundary point includes: Based on the preset height fitting function and the first mean height, the X-axis coordinates of the first boundary point are updated to obtain the updated first boundary point. The high-fit function is: ； Where a, b, c, and d are the coefficients of each term of the function, and h is the height of the first mean. The updated X-axis coordinates of the first boundary point; The step of calculating the scale correction ratio based on the updated coordinates of the first boundary point and the updated coordinates of the second boundary point includes: The distance between the left and right boundaries of the trapezoidal protrusion is calculated based on the updated coordinates of the first boundary point to obtain the width of the first trapezoidal boundary. The distance between the left and right boundaries of the trapezoidal protrusion is calculated based on the updated coordinates of the second boundary point to obtain the width of the second trapezoidal boundary. Calculate the width ratio of the first trapezoidal boundary width to the second trapezoidal boundary width to obtain the scale correction ratio; The point cloud of the object under test includes X-scale data, X-position data, and Z-axis height data. The step of aligning the binocular contour data according to the scale correction ratio, the average offset, and the height difference to obtain the point cloud of the object under test includes: An X-axis scale correction model is constructed based on the scale correction ratio; an X-axis position correction model is constructed based on the average offset; and a Z-axis height correction model is constructed based on the height difference. The binocular contour data are input into the X-axis scale correction model, the X-axis position correction model, and the Z-axis height correction model, respectively, to obtain the X-scale data, X-position data, and Z-axis height data; The binocular contour data is input into the X-axis scale correction model, and the scale of the binocular contour data is corrected according to the X-axis scale correction model to obtain the X-scale data. The binocular contour data are input into the X-axis position correction model, and the position of the binocular contour data is corrected according to the X-axis position correction model to obtain the X position data. The binocular contour data are input into the Z-axis height correction model, and the height data of the binocular contour data is corrected according to the Z-axis height correction model to obtain the Z-axis height data.

6. An electronic device, characterized in that, The electronic device includes: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the contour correction calibration method based on binocular vision as described in any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that, It includes a data storage area and a program storage area. The data storage area stores the created data, and the program storage area stores the computer program. When the computer program is executed by the processor, it implements the contour correction and calibration method based on binocular vision as described in any one of claims 1 to 4.

Images