Three-dimensional scanning method and three-dimensional scanning device

Abstract

The present disclosure relates to a kind of three-dimensional scanning device and three-dimensional scanning method, the three-dimensional scanning device includes: at least two image acquisition sensors, for acquiring the image of the surface of the object to be measured;Binocular vision measurement calculator, for the three-dimensional reconstruction of the surface feature of the object to be measured in the image;Global error control calculator, for the surface feature of the object to be measured in the image is calculated;Scale calculator, for determining scale based on the three-dimensional data set output by the binocular vision measurement calculator and the global error control calculator, to restore the object to be measured.The technical scheme provided by the embodiments of the present disclosure integrates photogrammetry and three-dimensional scanning, without the need for encoding points and high-cost and large-space-consuming scale physical objects, thereby controlling the global error while reducing the cost and complexity of the overall solution, so that the structure of three-dimensional scanning device is simple, easy to carry.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of three-dimensional scanning, and in particular to a three-dimensional scanning method and a three-dimensional scanning device. BACKGROUND

[0002] Optical three-dimensional scanning systems are commonly used fast three-dimensional measurement devices. In the industrial detection field, optical three-dimensional scanning is gradually becoming a mainstream detection technology. Industrial optical three-dimensional scanning technology is mainly divided into two categories according to the scanning method: fixed three-dimensional scanning and handheld three-dimensional scanning. The fixed three-dimensional scanning has the advantages of area array scanning, single-frame scanning can obtain three-dimensional data of the entire visible area, and single measurement efficiency and measurement accuracy are high. However, the fixed three-dimensional scanning device is usually heavy, inconvenient to use, and the manual operation experience is poor. The handheld three-dimensional scanning has the advantages of flexible and convenient operation, and is more suitable for measuring workpieces of various shapes and sizes. In particular, the handheld three-dimensional scanning device using a laser light source also has good adaptability to complex lighting, material, color, etc.

[0003] With the increasing improvement of industrial manufacturing level and quality control demand, three-dimensional scanning technology is also rapidly developing. Improving scanning accuracy is one of the focuses of this technical field. The existing laser three-dimensional scanning device has relatively reliable scanning accuracy in most small workpiece scanning application fields. However, when facing large workpiece scanning, the cumulative error is large due to the limitation of the technical principle, and it is difficult to independently complete the scanning task or meet the accuracy index. Therefore, it is usually necessary to cooperate with other global error control measurement devices, such as photogrammetry systems, that is, to use coding points, scales, and marker points to use multi-view geometry principle to obtain the object frame coordinates formed by the marker points to control the global error, which greatly increases the cost and complexity of the overall solution. SUMMARY

[0004] In order to solve the above technical problems or at least partially solve the above technical problems, the present disclosure provides a three-dimensional scanning method and a three-dimensional scanning device.

[0005] The present disclosure provides a three-dimensional scanning device, comprising:

[0006] At least two image acquisition sensors for acquiring images of the surface of the object to be measured;

[0007] A binocular vision measurement calculator for three-dimensional reconstruction of the surface features of the object to be measured in the images;

[0008] A global error control calculator for calculating the surface features of the object to be measured in the images;

[0009] A scale calculator for determining a scale based on the three-dimensional data sets output by the binocular vision measurement calculator and the global error control calculator to restore the object to be measured.

[0010] In some embodiments: the image acquisition sensor is configured to acquire images of the surface of the object to be measured at a first working distance during a photogrammetry phase;

[0011] the image acquisition sensor is further configured to acquire images of the surface of the object to be measured at a second working distance during a three-dimensional scanning phase;

[0012] wherein the second working distance is less than or equal to the first working distance.

[0013] In some embodiments, the binocular vision measurement calculator is configured to perform three-dimensional reconstruction on the marked points in the acquired images to determine a first coordinate set during the photogrammetry phase, and to perform three-dimensional reconstruction on the acquired images to determine a fourth coordinate set and point cloud data during the three-dimensional scanning phase.

[0014] the global error control calculator is configured to perform photogrammetry sparse beam adjustment calculation on the marked points in the acquired images to obtain a second coordinate set during the photogrammetry phase.

[0015] In some embodiments, the scale calculator includes a scale estimation calculator and a scale optimization calculator; the scale includes a first scale and a second scale.

[0016] the scale estimation calculator is configured to determine the first scale based on the first coordinate set and the second coordinate set.

[0017] determine a third coordinate set based on the second coordinate set and the first scale.

[0018] the scale optimization calculator is configured to determine the second scale based on the fourth coordinate set and the third coordinate set.

[0019] determine a marked point coordinate set based on the third coordinate set and the second scale.

[0020] In some embodiments, the three-dimensional scanning device further includes a pattern projector.

[0021] the pattern projector is configured to project a structured light pattern onto the surface of the object to be measured during the three-dimensional scanning phase.

[0022] the binocular vision measurement calculator is further configured to perform three-dimensional reconstruction on the structured light images in the acquired images to obtain point cloud data.

[0023] In some embodiments, the three-dimensional scanning device further includes a marked point encoder.

[0024] The mark point encoder is configured to encode mark points, and the encoded mark points are configured to realize comparison and identity recognition of mark points with the same name.

[0025] In some embodiments, the mark point encoder is configured to encode the mark points in a multi-point spatial structure encoding mode.

[0026] The present disclosure also provides a three-dimensional scanning method, which is performed by any one of the three-dimensional scanning devices described above, and the three-dimensional scanning method comprises the following steps:

[0027] At least two image acquisition sensors acquire images of a surface of an object to be measured;

[0028] A binocular vision measurement calculator performs three-dimensional reconstruction on surface features of the object to be measured in the images;

[0029] A global error control calculator performs calculation on the surface features of the object to be measured in the images;

[0030] A scale calculator determines a scale based on three-dimensional data sets output by the binocular vision measurement calculator and the global error control calculator, so as to restore the object to be measured.

[0031] In some embodiments, the at least two image acquisition sensors acquiring images of a surface of an object to be measured comprises:

[0032] In a photogrammetry stage, the image acquisition sensors acquire images of a surface of an object to be measured at a first working distance;

[0033] In a three-dimensional scanning stage, the image acquisition sensors acquire images of a surface of an object to be measured at a second working distance;

[0034] In some embodiments, the second working distance is less than or equal to the first working distance.

[0035] In some embodiments, the binocular vision measurement calculator performing three-dimensional reconstruction on surface features of the object to be measured in the images comprises:

[0036] In the photogrammetry stage, the binocular vision measurement calculator performs three-dimensional reconstruction on mark points in the acquired images to determine a first coordinate set;

[0037] In the three-dimensional scanning stage, the binocular vision measurement calculator performs three-dimensional reconstruction on the acquired images to determine a fourth coordinate set and point cloud data;

[0038] The global error control calculator performing calculation on the surface features of the object to be measured in the images comprises:

[0039] In the photogrammetry stage, the global error control calculator performs photogrammetry sparse beam adjustment calculation on the mark points in the collected images to obtain a second coordinate set.

[0040] In some embodiments, the scale calculator comprises a scale estimation calculator and a scale optimization calculator; the scale comprises a first scale and a second scale; the scale calculator determines the scale based on the three-dimensional data set output by the binocular vision measurement calculator and the global error control calculator, to restore the object to be measured to include:

[0041] The scale estimation calculator is configured to determine the first scale based on the first coordinate set and the second coordinate set;

[0042] The third coordinate set is determined based on the second coordinate set and the first scale;

[0043] The scale optimization calculator is configured to determine the second scale based on the fourth coordinate set and the third coordinate set;

[0044] The mark point coordinate set is determined based on the third coordinate set and the second scale;

[0045] The mark point coordinate set is used to splice multiple frames of the point cloud data.

[0046] In some embodiments, at least part of the coordinate points in the fourth coordinate set are in a trusted space, and the second scale is determined using the coordinate points in the fourth coordinate set that are in the trusted space;

[0047] The trusted space is a space-based pyramid within the measurement space of the three-dimensional scanning device, and the mark points in the space-based pyramid satisfy that the alignment error, the back projection error and the photogrammetry accuracy of a preset statistical proportion of the mark points are in the same order of magnitude.

[0048] In some embodiments, the three-dimensional scanning device further comprises a pattern projector, and the three-dimensional scanning method further comprises:

[0049] In the three-dimensional scanning stage, the pattern projector projects a structured light pattern onto the surface of the object to be measured;

[0050] The binocular vision measurement calculator performs three-dimensional reconstruction on the structured light images in the collected images to obtain point cloud data.

[0051] In some embodiments, the three-dimensional scanning device further comprises a mark point encoder, and the three-dimensional scanning method further comprises:

[0052] The mark point encoder encodes the mark points, and the encoded mark points are used to realize comparison and identity recognition of the mark points with the same name.

[0053] In some embodiments, the marker point encoder encodes the marker points in a multi-point spatial structure encoding manner.

[0054] Compared with the prior art, the technical solutions provided by the embodiments of the present disclosure have the following advantages:

[0055] The three-dimensional scanning device provided by the embodiments of the present disclosure acquires images of the surface of the object to be measured by using at least two image acquisition sensors, and performs three-dimensional reconstruction and calculation on the surface features of the object to be measured in the images by using a binocular vision measurement calculator and a global error control calculator respectively. On this basis, a scale calculator determines a scale based on the three-dimensional data set obtained through the three-dimensional reconstruction and calculation, so as to restore the object to be measured. The global error control calculator can be used to control the global error and make the global error smaller. Meanwhile, the scale determined by the scale calculator can be equivalent to the scale object (i.e. the scale) in the conventional measurement, so as to realize the conversion of the actual coordinates of each marker point in the determined space. Therefore, in this method, the scale can be used to replace the scale in the transmission photogrammetry scheme, so as to control the global error, simplify the overall solution, and reduce the cost of the three-dimensional scanning device. BRIEF DESCRIPTION OF DRAWINGS

[0056] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and serve to explain the principles of the present disclosure together with the specification.

[0057] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure or the prior art, the accompanying drawings needed to be used in the embodiments or the prior art description will be briefly introduced. Obviously, for those skilled in the art, other drawings can also be obtained based on these drawings without any creative effort.

[0058] Figure 1 FIG. 1 is a structural schematic diagram of a three-dimensional scanning device according to an embodiment of the present disclosure;

[0059] Figure 2 FIG. 2 is a structural schematic diagram of another three-dimensional scanning device according to an embodiment of the present disclosure;

[0060] Figure 3 FIG. 3 is a structural schematic diagram of still another three-dimensional scanning device according to an embodiment of the present disclosure;

[0061] Figure 4 FIG. 4 is a structural schematic diagram of still another three-dimensional scanning device according to an embodiment of the present disclosure;

[0062] Figure 5 FIG. 5 is a flow schematic diagram of a three-dimensional scanning method according to an embodiment of the present disclosure;

[0063] Figure 6 a flowchart of another three-dimensional scanning method of an embodiment of the present disclosure;

[0064] Figure 7 a flowchart of another three-dimensional scanning method of an embodiment of the present disclosure;

[0065] Figure 8 a flowchart of a first scale determination method of an embodiment of the present disclosure;

[0066] Figure 9 a flowchart of another first scale determination method of an embodiment of the present disclosure;

[0067] Figure 10 a flowchart of a second scale determination method of an embodiment of the present disclosure. DETAILED DESCRIPTION

[0068] In order to more clearly understand the above-mentioned purposes, features and advantages of the present disclosure, the schemes of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features in the embodiments can be combined with each other without conflict.

[0069] In the following description, many specific details are set forth in order to fully understand the present disclosure, but the present disclosure can also be implemented in other ways different from those described herein; obviously, the embodiments in the description are only some of the embodiments of the present disclosure, not all the embodiments.

[0070] In the background art, the photogrammetry principle is introduced into the three-dimensional scanning device, one of the cameras (i.e. image acquisition sensor) of the three-dimensional scanning device is used as a photogrammetry acquisition device, and the traditional photogrammetry technology is used, i.e. using coded points, scales, marker points, and using multi-view geometry principle to obtain the object frame coordinates composed of marker points to control the global error.

[0071] The advantage of the solution of the traditional photogrammetry system matched with the three-dimensional scanning system is reliable precision, but the defect is that the system cost and complexity of the solution are greatly increased. Specifically, the following deficiencies still exist:

[0072] Firstly, the system needs to be matched with auxiliary tools such as coded points and scales, and the auxiliary tools are also additional costs, especially large-size high-precision scales, which still have high costs;

[0073] Secondly, the size of the scale is relatively long, usually at least about 1m, which is not portable, and destroys the product characteristics of the handheld scanning system itself being small and portable;

[0074] Third, the system needs to arrange coded points on the object, especially large workpieces may need hundreds or thousands of coded points, which increases the complexity of the process and reduces the operation efficiency.

[0075] Fourth, the system uses traditional photogrammetry technology, i.e. single camera photogrammetry, which is difficult to perform real-time three-dimensional measurement due to the limitation of the principle. When used by users, there is no intuitive data display, and it is impossible to determine whether the shooting method and the number of images can guarantee the success of the measurement, calculation or whether the precision index can be met. Therefore, users have to collect a large number of redundant images to ensure success, which greatly reduces the measurement efficiency, and the robustness and precision of the measurement result are related to the user's experience, the measurement reliability is insufficient, and the user's learning cost is high.

[0076] In summary, the existing three-dimensional scanning technology with global error control generally has problems such as complex system, inconvenience, high cost, low measurement efficiency, difficult to guarantee reliability, and difficult to use.

[0077] In order to solve the above technical problems or at least partially solve the above technical problems, the embodiments of the present disclosure provide a three-dimensional scanning method and device with global error control, which makes the overall scheme simpler and more portable, reduces the system cost and user learning cost, and improves the measurement efficiency, measurement accuracy and reliability.

[0078] The three-dimensional scanning method provided by the embodiments of the present disclosure includes a photogrammetry stage and a three-dimensional scanning stage, and the scale can be determined based on a software algorithm, including a first scale and a second scale; the first scale is an estimated scale, which can realize preliminary determination of the actual coordinates of the mark points in space; and the second scale is an optimized scale, which can further optimize the actual coordinates to obtain high-precision space coordinates of the photogrammetry stage.

[0079] In some embodiments, under the condition that the measurement accuracy can be met, only the first scale can be used as a ruler without optimization of the scale. That is, the estimated scale determined by the photogrammetry stage is used as a ruler for positioning of the coordinate points to realize point cloud splicing, i.e. three-dimensional scanning of the object to be measured.

[0080] The three-dimensional scanning method and device provided by the embodiments of the present disclosure will be described below. Figures 1-10 The three-dimensional scanning device and method provided by the embodiments of the present disclosure will be described below.

[0081] Exemplarily, Figure 1 The structure of a three-dimensional scanning device according to an embodiment of the present disclosure is shown in FIG. 1. Referring to FIG. 1, Figure 1The three-dimensional scanning device can include: at least two image acquisition sensors 110 for acquiring images of the surface of the object to be measured; a binocular vision measurement calculator 120 for three-dimensional reconstruction of the surface features of the object to be measured in the images; a global error control calculator 130 for calculating the surface features of the object to be measured in the images; and a scale calculator 140 for determining a scale based on the three-dimensional data sets output by the binocular vision measurement calculator 120 and the global error control calculator 130, to restore the object to be measured.

[0082] The number of image acquisition sensors 110 is at least two, which can be two, three or more, to acquire images of the surface of the object to be measured from different angles, to form data for three-dimensional reconstruction and calculation. For example, when the number of image acquisition sensors 110 is greater than two, any at least two of them can be used to acquire images of the surface of the object to be measured.

[0083] The images of the surface of the object to be measured can include marker points and structured light patterns, or only marker points. The three-dimensional data set includes a coordinate set representing the spatial coordinates of the marker points obtained by three-dimensional reconstruction based on the images. In the following, the first, second, third, fourth coordinate sets and the marker point coordinate set (i.e. the final marker point coordinate set corresponding to the actual size of the object) are distinguished. Based on the structured light pattern, point cloud data can be obtained by three-dimensional reconstruction, and the surface of the object to be measured can be restored by splicing multiple frames of point cloud data.

[0084] The binocular vision measurement calculator 120 can perform three-dimensional reconstruction of the surface features (including marker points and / or structured light patterns) of the object to be measured in the images based on the binocular vision principle, to obtain the corresponding three-dimensional data set (described in detail below). The global error control calculator 130 can perform three-dimensional reconstruction of the surface features (marker points) of the object to be measured in the images based on the photogrammetry principle and / or the mean statistical optimization principle, to obtain the corresponding three-dimensional data set. The scale calculator 140 can perform data calculation based on the three-dimensional data sets obtained above, to obtain the scale for restoring the object to be measured. The scale is equivalent to the scale in the traditional measurement scheme, which can replace the scale in the traditional measurement scheme. The three-dimensional data set calculated by the global error control calculator 130 is a dimensionless coordinate set, which differs from the actual spatial coordinates of the marker points by a scale. The scale calculated by the scale calculator 140 is the scale that differs between the dimensionless coordinate set and the actual spatial coordinates of the marker points. The dimensionless coordinate set is converted into the marker point coordinate set (representing the actual spatial coordinates of the marker points) by the scale, to restore the real coordinates of the marker points and the real size between the marker points. Therefore, the three-dimensional scanning device can not need to use a scale, while controlling the global error, simplifying the overall solution and reducing the cost of the three-dimensional scanning device.

[0085] The global error control calculator 130 calculates the spatial relative relationship between the marked points in the surface features, i.e., the spatial relative coordinates, by using the photogrammetry principle, to control the global measurement error.

[0086] Exemplarily, the three-dimensional scanning device (which can also be referred to herein as "device" or "system" for short) can be a handheld scanning device, thus facilitating the portability and application flexibility.

[0087] The handheld scanning device includes two image acquisition sensors and a pattern projector. The images acquired by the two image acquisition sensors can be transmitted to the binocular vision measurement calculator for three-dimensional reconstruction based on the binocular vision measurement principle, and can also be transmitted to the global error control calculator for calculation based on the photogrammetry principle.

[0088] In some embodiments, the image acquisition sensor 110 is configured to acquire images of the surface of the object to be measured at a first working distance in a photogrammetry stage; and the image acquisition sensor 110 is also configured to acquire images of the surface of the object to be measured at a second working distance in a three-dimensional scanning stage; wherein the second working distance is smaller than the first working distance.

[0089] In the first working distance, the three-dimensional scanning device can acquire images at the farthest or relatively farthest end of the working distance to achieve image acquisition in a larger field of view range.

[0090] Exemplarily, the system acquires images at the farthest end of the working distance to obtain a single acquisition area as large as possible, thus reducing the number of data splicing and the cumulative error caused by data splicing.

[0091] Exemplarily, the single acquisition area can be 9 times or other multiples of the normal scanning area, which can be set according to the requirements of the three-dimensional scanning method and device, and can be determined by the working parameters of the three-dimensional scanning device, and the embodiments of the present disclosure are not limited thereto. Specifically, the global error control calculator 130 uses the system to acquire images at the farthest end of the working distance to obtain a single acquisition area as large as possible (e.g., 9 times of the normal scanning area), and performs three-dimensional reconstruction on the marked points on the surface of the object according to the photogrammetry principle. Since the single photogrammetry area is much larger than the scanning range (scanning is usually performed at a close distance to obtain better point cloud details, and the single frame measurement area is reduced to improve the spatial resolution), the number of data splicing is reduced, and the cumulative error caused by splicing is reduced. The positioning point coordinate set obtained by photogrammetry calculation can be used to define the object coordinate system, and the overall measurement accuracy can be improved.

[0092] The first working distance is greater than the second working distance, i.e., the second working distance is less than the first working distance. In this step, the mark points on the surface of the object to be measured are collected at a closer distance within the trusted space of the three-dimensional scanning device to obtain mark points with higher accuracy in the trusted space.

[0093] It should be noted that in the three-dimensional scanning stage, the working distance of the three-dimensional scanning device is closer, and under the condition that the baseline distance between the image acquisition sensors is certain, the spatial distance measurement uncertainty of binocular measurement is much smaller than that of the real-time acquisition process (binocular long-distance measurement) in the aforementioned photogrammetry stage. In addition, the constraint of the trusted space can improve the measurement accuracy of the spatial distance of the mark points.

[0094] In other embodiments, the second working distance can also be set to be equal to the first working distance. At this time, due to the large number of images collected, the scanning error can be reduced by using average data. In addition, the single-frame images collected at similar positions in the three-dimensional scanning stage and the photogrammetry stage do not completely overlap, and the corresponding data can be mutually compensated, which can also reduce the scanning error.

[0095] In some embodiments, the binocular vision measurement calculator 120 is configured to perform three-dimensional reconstruction on the mark points in the collected images in the photogrammetry stage to determine the first coordinate set, and perform three-dimensional reconstruction on the collected images in the three-dimensional scanning stage to determine the fourth coordinate set and the point cloud data. The global error control calculator 130 is configured to perform photogrammetry sparse beam method adjustment (i.e., photogrammetry principle) calculation on the mark points in the collected images in the photogrammetry stage to minimize the global error and obtain the second coordinate set.

[0096] In combination with the above, in the photogrammetry stage, 2D images of the surface of the object to be measured can be collected. The 2D images contain mark points, and the 2D images reconstruct the first coordinate set of the mark points based on the first measurement principle and the second coordinate set of the mark points based on the second measurement principle. In this embodiment, the same frame of 2D images reconstructs the first coordinate set of the mark points based on the binocular vision measurement principle and the second coordinate set of the mark points based on the photogrammetry principle.

[0097] The three-dimensional reconstruction of the mark points based on the binocular vision measurement principle can be referred to as the first stage of photogrammetry, which is a real-time measurement stage. The three-dimensional reconstruction result of the mark points can be displayed and monitored in real time. The first coordinate set determined in this stage is used to determine the first scale.

[0098] The three-dimensional reconstruction based on the photogrammetry principle refers to a process of optimizing calculation of collected images by using multi-view geometry principle and bundle adjustment to obtain relative coordinates of the marking points in space. The three-dimensional reconstruction of the marking points based on the photogrammetry principle can be referred to as a second stage of the photogrammetry, which is an optimization calculation stage. In this stage, a normalized calculation, dimensionless relative coordinate set of the marking points, i.e., a second coordinate set, is obtained. That is, in the second coordinate set, the relative positions of the marking points are relatively accurate, and the difference between the second coordinate set and the actual space coordinates is a scale.

[0099] In some embodiments, a scale can be determined in combination with the first coordinate set and the second coordinate set, i.e., the scale is estimated. That is, in the photogrammetry process, the distance between the marking points obtained in real time based on the binocular vision measurement principle is used as a reference to calculate the scale.

[0100] In some embodiments, a scale can be determined in combination with the first coordinate set and the second coordinate set, i.e., the scale is estimated. That is, in the photogrammetry process, the distance between the marking points obtained in real time based on the binocular vision measurement principle is used as a reference to calculate the scale.

[0101] Exemplarily, a plurality of different pairs of points in the first coordinate set can be taken to determine a first distance set, and correspondingly, a second distance set can be determined by using the corresponding points in the second coordinate set. By comparing a corresponding group, two groups or multiple groups of distances in the first distance set and the second distance set, the above-mentioned scale (i.e., the estimated scale) can be obtained. In this case, the corresponding points (i.e., the homonymous points) in the first coordinate set and the second coordinate set are determined by the marking point encoder described below.

[0102] In this way, by collecting the marking points on the surface of the object to be measured at a first working distance in the photogrammetry stage, and by reconstructing the collected marking points based on the binocular vision measurement principle and the photogrammetry principle respectively, the first coordinate set and the second coordinate set are obtained, and the scale is determined based on the first coordinate set and the second coordinate set. The scale can be equivalent to the scale in the conventional measurement. Therefore, in the three-dimensional scanning method, the scale can be used to replace the scale in the conventional photogrammetry scheme, which simplifies the overall solution and reduces the cost of the three-dimensional scanning device while controlling the global error.

[0103] On this basis, the scale can also be optimized, and the scale includes an estimated scale (i.e., a first scale) and an optimized scale (i.e., a second scale) to improve the accuracy of the scale. The following will be described exemplarily. Figure 2

[0104] In some embodiments, Figure 2 is another structure schematic diagram of a three-dimensional scanning device according to the embodiments of the present disclosure. In the Figure 1 above, reference can be made to​Figure 2 The scale calculator 140 comprises a scale estimation calculator 141 and a scale optimization calculator 142; the scale comprises a first scale and a second scale; the scale estimation calculator 141 is configured to determine the first scale based on the first coordinate set and the second coordinate set; determine a third coordinate set based on the second coordinate set and the first scale; the scale optimization calculator 142 is configured to determine the second scale based on the fourth coordinate set and the third coordinate set; determine the coordinate set of the marking points based on the third coordinate set and the second scale.

[0105] The step of determining the scale can comprise:

[0106] Step one: determining the first scale based on the first coordinate set and the second coordinate set.

[0107] In combination with the above, the first scale is the scale determined in the photogrammetry stage.

[0108] Step two: determining the third coordinate set based on the first scale and the second coordinate set.

[0109] In this step, the third coordinate set can be determined by using the more accurate spatial relative position relationship in the second coordinate set and the first scale determined in step one. The third coordinate set can not only accurately represent the spatial relative position relationship between any two marking points but also represent the distance value between any two marking points.

[0110] Exemplarily, this step can comprise taking the coordinate of a point in the second coordinate set as a reference, taking the azimuth and distance between points in the second coordinate set as a basis, and on this basis, multiplying the coordinate of each point by the first scale to obtain the coordinate set representing the actual coordinate of the marking point, i.e., to obtain the third coordinate set. It should be noted that when the scale only comprises the estimated scale, the third coordinate set is the coordinate set of the marking points.

[0111] In the photogrammetry stage, the second coordinate set determined based on the photogrammetry principle accurately describes the relative coordinate relationship between the marking points, but the scale is an estimated value because the reference selected is the first coordinate set, i.e., the coordinate set obtained by three-dimensional measurement and reconstruction at a long distance using the binocular vision measurement principle. Since the baseline between the image acquisition sensors of the three-dimensional scanning device is usually narrow and the working distance is large, the uncertainty of spatial measurement increases, resulting in that the dimensional accuracy of the first coordinate set can only reach the accuracy level of general three-dimensional scanning. Therefore, the scale estimated value needs to be further optimized to reach the photogrammetry accuracy level. That is, steps three and four are executed.

[0112] Step three: determining the second scale based on the third coordinate set and the fourth coordinate set.

[0113] The first scale and the second scale are superimposed to obtain a more accurate scale.

[0114] The third coordinate set can accurately represent the relative position relationship between any two points, and the fourth coordinate set can accurately represent the distance between any two marker points. The distance between the two points corresponding to the distance value determined by the third coordinate set exists, and the second scale can be determined by comparing the corresponding distance values. That is, the distance between any two points in the determined fourth coordinate set is determined, and the scale is calculated by the distance between the corresponding two points in the third coordinate set, so as to achieve the purpose of optimizing the scale, which will be described in detail below. The corresponding points (i.e. homonymous points) between the third coordinate set and the fourth coordinate set are encoded, identified and determined by the marker point encoder described below.

[0115] Then, step four can be performed, that is, the scale is determined based on the first scale and the second scale. The scale is a scale representing the distance ratio between the corresponding coordinate points of the second coordinate set and the actual space coordinate set of the marker points.

[0116] In this way, the measurement accuracy of the three-dimensional scanning device can be improved by several times. Taking a typical current industrial handheld scanning device as an example, the volume accuracy can be improved from 0.06-0.1 mm / m to 0.015-0.03 mm / m, which is comparable to the current typical industrial photogrammetry system. However, the three-dimensional scanning method provided by the present disclosure determines the first scale and the second scale based on the marker points, that is, the size measurement capability of the binocular vision measurement system at different working distances is used to estimate and optimize the scale of the photogrammetry result, which can replace the scale in the traditional photogrammetry scheme, so that a physical scale is not needed, making the structure of the three-dimensional scanning device simpler and the cost lower.

[0117] In the three-dimensional scanning device provided by the present disclosure, the binocular vision measurement calculator 120 estimates and optimizes the scale of the result obtained by the global error control calculator 130 based on the size measurement capability at different working distances (i.e. the first working distance and the second working distance), which can replace the scale in the traditional photogrammetry scheme, and at the same time, the global error is controlled, the entire solution is simplified, and the cost of the three-dimensional scanning device is reduced.

[0118] In other embodiments, under the condition of meeting the accuracy requirement, the first scale can be determined based on the first coordinate set and the second coordinate set; the second scale can be determined based on the third coordinate set and the fourth coordinate set; and the final scale can be obtained by superimposing the first scale and the second scale. The present disclosure does not limit this.

[0119] In some embodiments, at least part of the coordinate points in the fourth coordinate set are in the trusted space, and the scale is determined by using the coordinate points in the fourth coordinate set that are in the trusted space, which can further improve the accuracy of the scale, thereby facilitating the improvement of the three-dimensional scanning accuracy.

[0120] The trusted space is a space frustum in the measurement space of the three-dimensional scanning device. The mark points in the space frustum satisfy that the alignment error and the back projection error of a preset statistical proportion of the mark points are in the same order of magnitude as the photogrammetry accuracy.

[0121] The trusted space is usually related to the physical properties of the optical system and the mathematical model used in the calibration calculation, such as the collimation of the optical path, the imaging quality of the lens, and the distortion model. The space is a part of the continuous space in the measurement space of the device, and is usually a space frustum. The space can be marked according to the criterion that the alignment error and the back projection error of a certain statistical proportion (such as more than 90%) of the mark points are in the same order of magnitude as the photogrammetry accuracy.

[0122] During calibration, different poses are used to collect the mark points on the calibration board, so that the mark points are accumulated in the entire three-dimensional measurement space of the three-dimensional scanning device in time. At this time, all the point coordinates that satisfy the accuracy distribution are approximately in a frustum-shaped space (i.e., the trusted space) in the device coordinate system. The geometric parameters of the frustum-shaped space are fitted and recorded for subsequent calculation.

[0123] For example, during calibration of the three-dimensional scanning device, the trusted space in the device coordinate system is calibrated. In the space, the space alignment error and the back projection error of most of the mark points (for example, the statistical proportion is equal to or greater than 90%) satisfy that the space alignment error and the back projection error are in the same order of magnitude as the photogrammetry accuracy.

[0124] For example, the accuracy of photogrammetry can be about 0.02 mm / m, and the accuracy of three-dimensional scanning can be more than 0.08 mm / m. The alignment error and the back projection error are in the same order of magnitude as the photogrammetry accuracy, which means that the alignment error and the back projection error are in the order of magnitude of 0.02 mm / m.

[0125] The back projection error is related to the chip size and the field of view size of the image acquisition sensor, and the like, which are not described herein.

[0126] Therefore, based on the reliable mark point distance set in the fourth coordinate set that is in the trusted space, the distance between two mark points in a single frame of data is determined, which can exclude the influence of the cumulative error, has higher accuracy, and is closer to the true value. In this way, the overall accuracy of the scale can be improved by superimposing the first scale and the second scale.

[0127] On the basis of the above-mentioned embodiments, in order to increase the features of the surface of the object to be measured, a structured light pattern can also be projected onto the surface of the object to be measured, and the following is an example description in combination with Figure 3

[0128] In some embodiments, Figure 3 FIG. 1 is a schematic diagram of a three-dimensional scanning device according to another embodiment of the present disclosure. Referring to FIG. 1, Figure 3 The three-dimensional scanning device can also include a pattern projector 150, which is configured to project a structured light pattern onto the surface of the object to be measured during the three-dimensional scanning phase. The binocular vision measurement calculator 120 is further configured to perform three-dimensional reconstruction on the structured light image in the acquired image to obtain point cloud data.

[0129] The pattern projector 150 can project a specific structured optical pattern onto the surface of the object to be measured to identify the features of the surface of the object to be measured.

[0130] For example, the pattern projector 150 can be a laser light source, a light-emitting diode light source, a halogen lamp light source, or other types of structured light sources.

[0131] Therefore, the binocular vision measurement calculator 120 can extract the 2D features of the object surface from the images acquired by the two image acquisition sensors 110 and perform three-dimensional reconstruction using the binocular stereo vision principle.

[0132] The structured light pattern is a projection pattern with a geometric pattern, which is used to mark the features of the surface of the object to be measured to achieve feature matching of the surface of the object to be measured.

[0133] For example, the shape of the projection pattern can be a stripe and / or a speckle, such as a sinusoidal stripe or a binary stripe, and a black-and-white snowflake pattern or a pattern similar to a two-dimensional code.

[0134] Thereafter, based on the binocular vision measurement principle, the structured light pattern and the marker points are reconstructed in three dimensions to obtain a fourth coordinate set and point cloud data. The fourth coordinate set and the point cloud data reconstructed in the same frame of image are in the same coordinate system, and the positional relationship is determined.

[0135] In this step, the structured light pattern is used as a feature identifier of the surface of the object to be measured, and the binocular stereo vision principle (i.e., the binocular vision measurement principle) is used to perform three-dimensional reconstruction to obtain point cloud data (i.e., a point set). At the same time, the marker points are used as identification points for subsequent point cloud splicing to facilitate the alignment and splicing of multiple frames of point cloud data.

[0136] Thereafter, based on the scale and the second coordinate set, the coordinate set of the marker points is determined.

[0137] ​That is, on the basis that the second coordinate set can accurately represent the spatial relative positions of each marker point, the marker point coordinate set representing the accurate positions of the marker points in the spatial coordinate system is converted through a scale, to prepare for the accurate splicing of subsequent point cloud data.

[0138] Finally, the multi-frame point cloud data is spliced based on the marker point coordinate set.

[0139] It should be noted that the global marker points on the surface of the measured object generally need to be measured at multiple angles, and a second coordinate set is obtained at each angle. Only the local marker points on the surface of the measured object are obtained, and therefore, a marker point coordinate set is obtained at each angle. After the marker point coordinate sets obtained at multiple angles are unified in the same coordinate system based on the same marker points, the marker point coordinate set of the global marker points is obtained. The process of unifying the marker point coordinate sets obtained at multiple angles in the same coordinate system based on the same marker points includes: determining a point in the marker point coordinate set of one local marker point, determining the corresponding marker point (i.e., the same marker point) in the marker point coordinate set of another local marker point based on the coding information (e.g., the length of the side of the polygon formed by the local marker point and other marker points in the neighborhood, the included angle, etc.) of the marker point, and unifying the marker point coordinate sets of the two local marker points in the same coordinate system based on the same marker points through rigid transformation.

[0140] The marker points (i.e., the fourth coordinate set) in the single-frame point cloud data are used as the identification points for splicing the point cloud data. The fourth coordinate set is aligned with the same marker points in the marker point coordinate set, and the splicing of the point cloud data can be achieved. The same marker points (i.e., the same points and corresponding points) in the fourth coordinate set and the marker point coordinate set are determined through the marker point encoder described below.

[0141] Based on this, the point cloud data obtained through sequential scanning is sequentially spliced by taking the marker point coordinate set as the frame point (splicing reference), and a complete scanning image, i.e., the three-dimensional profile of the surface of the measured object, can be obtained. Since the marker point coordinate set is obtained based on the second coordinate set obtained in the remote photogrammetry stage, the splicing of the multi-frame point cloud data according to the marker point coordinate set can reduce the cumulative error of the splicing of the multi-frame point cloud data, obtain point cloud data with high overall accuracy, and the measurement accuracy of each point in the point cloud data is also high.

[0142] At this point, the scanning of the surface of the measured object with high accuracy is completed.

[0143] In some embodiments, the depth of field of the image acquisition sensor 110 is greater than the preset depth of field, or the image acquisition sensor 110 is a zooming and variable depth of field acquisition sensor.

[0144] In the image acquisition sensor 110, a large depth of field acquisition sensor can be used. In the case where the imaging lens focal length, working distance and aperture value are known, the larger the chip size, the larger the depth of field. According to the optical imaging depth of field calculation method, the minimum sensor chip size can be calculated according to the known focal length, working distance, aperture value and the required depth of field size, and the sensor can meet the large depth of field working requirement. In this way, the image acquisition sensor can be used for close-range scanning to obtain high-accuracy three-dimensional data, and can also be used for long-range photogrammetry to obtain large-format three-dimensional data and reduce the number of splicing. In the three-dimensional scanning stage, close-range scanning is used to obtain high-accuracy fourth coordinate set and point cloud data, and the point cloud data has good details and high spatial resolution. In the photogrammetry stage, long-range measurement is used, and a single frame can obtain large-format first coordinate set and second coordinate set. The entire system only uses two image sensors to realize one machine with multiple functions, i.e., point cloud scanning and photogrammetry.

[0145] At the same time, the image acquisition sensor 110 can also use a variable focal length and variable depth of field acquisition sensor, for example, a variable focal length and variable depth of field acquisition sensor equipped with a variable focal length and variable depth of field (such as an electric motor lens, a liquid lens, etc.) can meet different depth of field working requirements to realize one machine with multiple functions, i.e., point cloud scanning and photogrammetry.

[0146] In the above embodiment, the marker point can be a coded point or a non-coded point. When the marker point is a non-coded point, i.e., the marker point itself has no code and no unique feature identifier, it can be called a common marker point. Based on this, the common marker point can be coded to replace the coded point required in the traditional photogrammetry scheme, thereby eliminating the process of additionally pasting coded points on the workpiece, reducing the process complexity and improving the work efficiency. The following will be described in conjunction with Figure 4 for an exemplary description.

[0147] In some embodiments, Figure 4 is a structural schematic diagram of another three-dimensional scanning device of the embodiment of the present disclosure. Referring to Figure 4 The three-dimensional scanning device further comprises a marker point encoder 160; the marker point encoder 160 is used for encoding the marker point, and the encoded marker point is used for realizing the comparison and identity recognition of the marker points with the same name.

[0148] In the marker point encoder 160, a multi-point spatial structure encoding mode can be realized, and the target marker point and other marker points in the neighborhood form a spatial polygon, such as a triangle or a spatial quadrilateral. The polygon side length and the included angle are used as the encoding information of the target point, and are used for the comparison and identity recognition of the marker points with the same name in the dynamic measurement process.

[0149] In the photogrammetry stage, the binocular vision measurement principle is used for photogrammetry, and the spatial three-dimensional coordinates of the ordinary mark points can be obtained in real time. Thus, the spatial geometric relationship of the mark points can be established, which is equivalent to realizing the spatial coding of the mark points. Thus, the spatial geometric coding technology of the ordinary mark points can be used instead of the special coding points.

[0150] In the three-dimensional scanning device provided by the embodiments of the present disclosure, the scale estimator 141 is used in the photogrammetry stage, the binocular vision measurement principle is used, the calibrated internal and external parameters of the binocular system are used for real-time three-dimensional reconstruction of the mark points, the matching mark points are identified by the mark point encoder 160, data registration is performed, and thus the real-time dynamic display and monitoring of the photogrammetry three-dimensional reconstruction result can be realized. After the dynamic measurement is completed, the internal parameters of the image acquisition sensor and the mark point data (mark point coding ID, image coordinates, etc.) of all single-frame sampling are used for optimization calculation by using the multi-view geometric principle and the bundle adjustment method, and the mark point spatial relative coordinate set is obtained.

[0151] In this process, the photogrammetry without a scale obtains a certain normalized calculation, a dimensionless mark point spatial relative coordinate set A (i.e., the second coordinate set), and the difference between the actual spatial coordinates is a scale. The distance between the mark points obtained by the binocular vision principle in real time in the photogrammetry process can be selected as a reference to calculate the scale.

[0152] For example, one implementation is that if the mark point coordinate set obtained in real time is B (i.e., the first coordinate set), the distance L between any two points in A is selected as a reference to calculate the scale. A The distance L between the corresponding two points in B is found. B L B / L A The first scale is obtained. After the first scale is used to convert A, the coordinate set in the actual scale space is transformed into A*, which is the third coordinate set.

[0153] In the photogrammetry stage, the spatial relative coordinate set A obtained by the photogrammetry accurately describes the relative coordinate relationship between the mark points, but the first scale is an estimated value. The reference used is the coordinate set B, i.e., the three-dimensional measurement at a long distance by using the binocular vision principle. Since the baseline between the image acquisition sensors of the three-dimensional scanning device is usually narrow, and the working distance is large, the spatial measurement uncertainty increases, which leads to the fact that the size accuracy of the coordinate set B can only reach the accuracy level of general three-dimensional scanning. Therefore, the scale estimated value must be further optimized to reach the photogrammetry accuracy level.

[0154] The above scale optimization calculator 142 is used in the three-dimensional scanning stage, and specifically can include: marking a reliable space in a device coordinate system during system calibration, and the space satisfies that the spatial alignment error and the back projection error of most feature points (such as more than 90% of statistical scales) during calibration are equivalent to the photogrammetry accuracy; during the three-dimensional scanning process, a reliable marker point distance set in all reliable spaces is calculated and recorded in real time; a statistical method is used to optimize the scale, a large number of space distances in the reliable coordinate set recorded during the scanning process are sampled to optimize the third coordinate set, and an accurate and photogrammetry-level-precision marker point coordinate set is obtained.

[0155] The statistical method is used to optimize the scale, that is, a large number of samples in the reliable marker point distance set are taken, the scale is calculated with the corresponding point distance in the coordinate set A*, the statistical method (such as mean value) is used to optimize the scale, the random error is reduced, and the precision of the scale calculation is improved. Since the measurement accuracy is determined by the accuracy and the precision, the accuracy and the precision are simultaneously optimized to improve the accuracy.

[0156] Exemplarily, the measurement accuracy of the three-dimensional scanning device of the embodiment can be improved by several times compared with the three-dimensional scanning device in the prior art. Taking a current typical industrial handheld scanning device as an example, the volume accuracy can be increased from 0.06-0.1 mm / m to 0.015-0.03 mm / m, which is equivalent to the current typical photogrammetry system.

[0157] In some embodiments, each marker point has the same shape and size.

[0158] In this way, each marker point is the same, and no differentiated design and setting is provided except for the spatial position, so that the design and manufacturing difficulty of the marker point can be reduced.

[0159] Exemplarily, the marker point can be a common circular marker point or a marker point of other shapes known to those skilled in the art, and the embodiment of the disclosure is not limited in this regard.

[0160] In other embodiments, other coding methods can also be used to code the marker points, and the embodiment of the disclosure is not limited in this regard.

[0161] On the basis of the above embodiment, the working process of the three-dimensional scanning device can include:

[0162] In the photogrammetry stage, the pattern projector 150 does not work, and the two image acquisition sensors 110 in the handheld scanning device are used to acquire an image of the surface of the object to be measured at a first working distance, and the image includes the marker points.

[0163] The binocular vision measurement calculator 120 performs three-dimensional reconstruction on the marker points acquired in the photogrammetry stage to determine the first coordinate set.

[0164] The global error control calculator 130 performs three-dimensional reconstruction on the mark points collected in the photogrammetry stage to determine a second coordinate set.

[0165] The scale estimation calculator 141 determines a first scale based on the first coordinate set and the second coordinate set, and determines a third coordinate set based on the second coordinate set and the first scale.

[0166] In the three-dimensional scanning stage, the pattern projector 150 works to project a structured light pattern to the surface of the object to be measured, and the two image acquisition sensors 110 in the handheld scanning device acquire an image of the surface of the object to be measured at a second working distance, the image including mark points and the structured light pattern.

[0167] The binocular vision measurement calculator 120 performs three-dimensional reconstruction on the mark points collected in the three-dimensional scanning stage to determine a fourth coordinate set, and performs 2D feature extraction based on the structured light pattern, and performs three-dimensional reconstruction using the binocular stereo vision principle to obtain single-frame point cloud data.

[0168] The scale optimization calculator 142 determines a second scale based on the fourth coordinate set and the third coordinate set, and determines a mark point coordinate set based on the second scale and the third coordinate set (or can be based on the scale and the second coordinate set to determine the mark point coordinate set).

[0169] The point cloud splicing subunit (not shown in the figure) is used to splice multiple frames of point cloud data based on the mark point coordinate set. The three-dimensional scanning device provided by the embodiment of the disclosure has at least the following beneficial effects:

[0170] First, the photogrammetry and the three-dimensional scanning device are integrated.

[0171] The handheld scanning device provided by the embodiment of the disclosure highly integrates the photogrammetry function, simplifies, integrates and portabilizes the scheme of superimposing photogrammetry on three-dimensional scanning.

[0172] Second, dynamic photogrammetry.

[0173] The dynamic photogrammetry function can display the three-dimensional reconstruction result of the mark points in real time during the photogrammetry process, intuitively helping or guiding the user to judge the completeness of data acquisition, and the user does not need to collect a large number of redundant images to ensure the success rate. The measurement result is what you see, the robustness and precision are independent of the user's experience, so the measurement efficiency, reliability and learning cost of the user are improved.

[0174] Third, photogrammetry without coding points.

[0175] The photogrammetry function uses ordinary mark point space geometric coding technology instead of special coding points, which saves the process of additionally pasting coding points on the workpiece, reduces the process complexity, and improves the operation efficiency.

[0176] Fourth, no scale photogrammetry.

[0177] The embodiment of the present disclosure proposes that the size measurement capability of the binocular vision measurement system with different working distances in photogrammetry is used to estimate and optimize the scale of the photogrammetry result, replacing the scale in the traditional photogrammetry scheme, simplifying the entire solution and reducing the cost.

[0178] On the basis of the above-mentioned embodiments, the present disclosure also provides a three-dimensional scanning method which can be executed by any one of the three-dimensional scanning devices in the above-mentioned embodiments. Therefore, the three-dimensional scanning method also has the beneficial effects of the three-dimensional scanning device. The same parts can be understood with reference to the above explanation and description of the three-dimensional scanning device, which will not be repeated hereinafter.

[0179] Exemplarily, Figure 5 is a flowchart of a three-dimensional scanning method according to an embodiment of the present disclosure. With reference to Figure 5 , the three-dimensional scanning method comprises:

[0180] S510, at least two image acquisition sensors acquire images of the surface of the object to be measured.

[0181] S520, a binocular vision measurement calculator performs three-dimensional reconstruction on the surface features of the object to be measured in the images.

[0182] S530, a global error control calculator calculates the surface features of the object to be measured in the images.

[0183] S540, a scale calculator determines a scale based on the three-dimensional data set output by the binocular vision measurement calculator and the global error control calculator to restore the object to be measured.

[0184] In this way, based on the images of the surface of the object to be measured acquired by the image acquisition sensor, the binocular vision measurement calculator and the global error control calculator are respectively used to perform three-dimensional reconstruction and calculation on the surface features of the object to be measured in the images, and based on the obtained three-dimensional data set, the scale calculator determines the scale, so that the scale of the real object can be replaced.

[0185] In some embodiments, acquiring images can include acquiring images in the photogrammetry stage and the three-dimensional scanning stage respectively, that is, S510 can include: in the photogrammetry stage, the image acquisition sensor acquires images of the surface of the object to be measured at a first working distance; and in the three-dimensional scanning stage, the image acquisition sensor acquires images of the surface of the object to be measured at a second working distance; wherein the second working distance is smaller than the first working distance.

[0186] In this way, a larger single acquisition image can be obtained at the first working distance, so that the number of image splicing and the global error can be reduced.

[0187] In some embodiments, corresponding to the images acquired in the photogrammetry stage and the three-dimensional scanning stage in S510, the processing of the images can include: in the photogrammetry stage, the binocular vision measurement calculator performs three-dimensional reconstruction on the mark points in the acquired images to determine the first coordinate set; and in the three-dimensional scanning stage, the binocular vision measurement calculator performs three-dimensional reconstruction on the acquired images to determine the fourth coordinate set and the point cloud data.

[0188] Based on this, S530 can include: in the photogrammetry stage, the global error control calculator performs photogrammetry sparse beam adjustment calculation on the mark points in the acquired images to minimize the global error and obtain the second coordinate set.

[0189] In some embodiments, in combination with the above, the scale calculator includes a scale estimation calculator and a scale optimization calculator; and the scale includes a first scale and a second scale. Based on this, S540 can include:

[0190] The scale estimation calculator is configured to determine the first scale based on the first coordinate set and the second coordinate set;

[0191] The third coordinate set is determined based on the second coordinate set and the first scale;

[0192] The scale optimization calculator is configured to determine the second scale based on the fourth coordinate set and the third coordinate set;

[0193] The mark point coordinate set is determined based on the third coordinate set and the second scale;

[0194] The mark point coordinate set is used for splicing the plurality of frames of point cloud data.

[0195] In this way, the scale can be estimated and optimized using the three-dimensional data set, and the point cloud splicing can be completed, and the scanning accuracy is high and the convenience is good.

[0196] In some embodiments, at least part of the coordinate points in the fourth coordinate set are in a trusted space, and the second scale is determined using the coordinate points in the fourth coordinate set that are in the trusted space; and the trusted space is a space-based pyramid table in the measurement space of the three-dimensional scanning device, and the mark points in the space-based pyramid table satisfy: the alignment error, the back projection error and the photogrammetry accuracy of a preset statistical proportion of the mark points are in the same order of magnitude.

[0197] In this way, the accuracy of the scale can be further improved using the trusted space, so as to improve the scanning accuracy.

[0198] In some embodiments, the three-dimensional scanning device further comprises a pattern projector. Based on this, the three-dimensional scanning method can further comprise:

[0199] In the three-dimensional scanning stage, the pattern projector projects a structured light pattern to the surface of the object to be measured.

[0200] The binocular vision measurement calculator performs three-dimensional reconstruction on the structured light image in the collected image to obtain point cloud data.

[0201] In this way, the surface features of the object to be measured in the image are identified, the point cloud data is spliced, and the scanning accuracy is ensured.

[0202] In some embodiments, the three-dimensional scanning device further comprises a marker point encoder, and the three-dimensional scanning method further comprises:

[0203] The marker point encoder encodes the marker points, and the encoded marker points are used to realize the comparison and identity recognition of the marker points with the same name.

[0204] In this way, the marker points on the surface of the object to be measured can be set as non-encoded marker points, thereby simplifying the difficulty of setting the marker points.

[0205] In some embodiments, the marker point encoder encodes the marker points in a multi-point space structure encoding manner.

[0206] Based on the above embodiments, an optional execution flow of the three-dimensional scanning method provided by the embodiments of the present disclosure is exemplarily described.

[0207] Exemplarily, Figure 6 For the flowchart of another three-dimensional scanning method provided by the embodiments of the present disclosure, only the process of determining the estimated scale is shown. Referring to Figure 6 , the three-dimensional scanning method can comprise:

[0208] S611, in the photogrammetry stage, the marker points on the surface of the object to be measured are collected at a first working distance.

[0209] S612, based on the binocular vision measurement principle, the marker points are three-dimensionally reconstructed to determine a first coordinate set.

[0210] S613, based on the photogrammetry principle, the marker points are three-dimensionally reconstructed to determine a dimensionless second coordinate set.

[0211] In other embodiments, S613 can be performed before S612, or both can be performed in parallel, and the embodiments of the present disclosure are not limited thereto.

[0212] S614, based on the first coordinate set and the second coordinate set, an estimated scale is determined.

[0213] After that, the estimated scale is used to determine the coordinate set of the mark points representing the size, to assist the point cloud splicing, so as to complete the three-dimensional scanning process.

[0214] On the basis of the above-mentioned embodiments, the estimated scale determined in the photogrammetry stage can also be optimized. Optionally, the fourth coordinate set is determined in the three-dimensional scanning stage, and the scale is determined on the basis of the first coordinate set and the second coordinate set in combination with the fourth coordinate set. The following is an exemplary description. Figure 7

[0215] Exemplarily, Figure 7 A flowchart of another three-dimensional scanning method provided by the embodiments of the present disclosure is shown in the figure. On the basis of the above-mentioned embodiments, Figure 6 , the three-dimensional scanning method can further include: Figure 7

[0216] S621, in the three-dimensional scanning stage, the mark points on the surface of the object to be measured are collected at the second working distance.

[0217] S622, based on the binocular vision measurement principle, the mark points are three-dimensionally reconstructed to determine the fourth coordinate set.

[0218] In this step, the mark points collected in S621 are reconstructed based on the binocular vision measurement principle, and the fourth coordinate set can be determined, which prepares for the subsequent determination of the mark point interval to determine the scale.

[0219] S623, based on the first coordinate set, the second coordinate set and the fourth coordinate set, the scale is determined.

[0220] That is, after S611, S612, S613, S621 and S622, the coordinate set determined in the photogrammetry stage can be combined with the coordinate set determined in the three-dimensional scanning stage to determine the scale. This step is equivalent to the optimization of S614 in the above-mentioned embodiments. The scale obtained after the optimization has higher precision, which is beneficial to improving the three-dimensional scanning precision. Figure 6

[0221] In this three-dimensional scanning method, the structure light pattern can also be projected onto the surface of the object to be measured at the second working distance in the three-dimensional scanning stage, and the structure light pattern and the mark points on the surface of the object to be measured are collected.

[0222] That is, in the image of the surface of the object to be measured collected at the second working distance in S621, the structure light pattern projected onto the surface of the object to be measured is also included in addition to the mark points. This three-dimensional scanning method further includes that the point cloud three-dimensionally reconstructed by the structure light pattern is spliced based on the mark points, the characteristics of the surface of the object to be measured are represented, and the whole process of the three-dimensional scanning of the object to be measured is completed.

[0223] ​​​In the above embodiments, regarding the determination of the first scale and the second scale, the first scale can be determined according to a corresponding distance set in the three-dimensional data set associated with the scale, which will be described below in combination with Figures 8-10 Exemplary description is made.

[0224] Firstly, the following Figures 8-9 , the process of determining the first scale is described in detail.

[0225] In some embodiments, Figure 8 A flowchart of a first scale determination method of an embodiment of the present disclosure is shown. Referring to Figure 9 , the first scale determination method can include:

[0226] S210, any two points in the first coordinate set are taken, and the distance between the two points is determined as the first distance.

[0227] Exemplarily, any two points L B in the first coordinate set are taken.

[0228] S220, in the second coordinate set, two points corresponding to the two points taken in the first coordinate set are determined, and the distance between the two points is determined as the second distance.

[0229] Exemplarily, the two points corresponding to the two points taken in S210 in the second coordinate set are searched, and the distance between the two points is determined as L A .

[0230] S230, the ratio of the first distance to the second distance is determined as the first scale.

[0231] Exemplarily, the ratio of the first distance L B to the second distance L A in S210 can be represented as: L B / L A , which is the first scale.

[0232] On this basis, the second coordinate set is converted into a coordinate set in the actual scale space after being converted by the first scale, that is, a third coordinate set is obtained.

[0233] The above Figure 8 shows that, taking two points in the first coordinate set as a reference, the corresponding two points in the second coordinate set are searched, and the ratio of the distance between the two points is calculated as the first scale.

[0234] In other embodiments, two points in the second coordinate set can also be taken as a reference, the corresponding two points in the first coordinate set are searched, and the ratio of the distance between the two points is calculated as the first scale, which will be described below in combination with Figure 9 Exemplary description is made.

[0235] In some embodiments, Figure 9 A flowchart of another first scale determination method for embodiments of the present disclosure is shown. Referring to Figure 9 The first scale determination method can include:

[0236] S310, any two points in the second coordinate set are taken, and the distance between the two points is determined as the second interval.

[0237] Exemplarily, L A is taken as the distance between any two points in the second coordinate set.

[0238] S320, in the first coordinate set, two points corresponding to the two points taken in the second coordinate set are determined, and the distance between the two points is determined as the first interval.

[0239] Exemplarily, the two points in the first coordinate set corresponding to the two points taken in S310 are found, and the interval therebetween is determined as L B .

[0240] S330, the ratio of the first interval to the second interval is determined as the first scale.

[0241] Exemplarily, the ratio of the first interval L B to the second interval L A in S320 can be represented as: L B / L A , which is the first scale.

[0242] On this basis, the second coordinate set is converted into a coordinate set in the actual scale space after being converted by the first scale, i.e., a third coordinate set is obtained.

[0243] On the basis of the above Figure 8 and Figure 9 , a statistical method or other more accurate method can be used to determine the first scale to improve the accuracy of the first scale. For example, using a statistical method: more sampling data in the first coordinate set and the second coordinate set can be used to calculate the first scale, i.e., the first scale is optimized, which can include:

[0244] First, based on the corresponding multiple points in the first coordinate set and the second coordinate set, multiple first scales are determined.

[0245] Exemplarily, this step can be understood in combination with Figure 8 or Figure 9 , i.e., the corresponding any two points in the first coordinate set and the second coordinate set can be taken multiple times to determine multiple first scales based on the corresponding first interval and the second interval.

[0246] After that, the average value of the multiple first scales is determined as the optimized first scale.

[0247] Exemplarily, in this step, the plurality of scales determined in the step of determining the first scale can be averaged to determine the optimized first scale. Figure 8 or Figure 9

[0248] The optimized first scale is higher in accuracy relative to the scales determined in the step of determining the first scale and the step of determining the second scale, thereby facilitating improvement of the measurement accuracy of the three-dimensional scanning. Figure 8 and Figure 9 The scales determined in the step of determining the first scale and the step of determining the second scale are determined by using a single set of corresponding intervals, and thus are higher in accuracy, thereby facilitating improvement of the measurement accuracy of the three-dimensional scanning.

[0249] The above exemplarily illustrates the determination process of the first scale, including the optimization process thereof. The above Figures 8-9 Similarly, the second scale can be determined based on the third coordinate set and the fourth coordinate set, and the third coordinate set can be taken as the reference coordinate set, or the fourth coordinate set can be taken as the reference coordinate set. That is, the determination method of the second scale can include:

[0250] First, any two points in the third coordinate set are taken, and the distance between the two points is determined as the third interval.

[0251] Second, in the fourth coordinate set, two points corresponding to the two points taken in the third coordinate set are determined, and the distance between the two points is determined as the fourth interval.

[0252] Finally, the ratio of the fourth interval to the third interval is determined as the second scale.

[0253] Thus, the third coordinate set is taken as the reference coordinate set, and the corresponding interval is found in the fourth coordinate set to determine the second scale.

[0254] Alternatively, the determination method of the second scale can also include:

[0255] First, any two points in the fourth coordinate set are taken, and the distance between the two points is determined as the fourth interval.

[0256] Second, in the third coordinate set, two points corresponding to the two points taken in the fourth coordinate set are determined, and the distance between the two points is determined as the third interval.

[0257] Finally, the ratio of the fourth interval to the third interval is determined as the second scale.

[0258] Thus, the fourth coordinate set is taken as the reference coordinate set, and the corresponding interval is found in the third coordinate set to determine the second scale.

[0259] On the basis of the above-illustrated determination method of the second scale, the second scale can also be optimized. The following will be exemplarily illustrated in combination with Figure 10

[0260] ​​In some embodiments, Figure 10 A flowchart of a second scale determination method of an embodiment of the present disclosure is shown. Referring to Figure 10 The second scale determination method can include:

[0261] S410, taking multiple sets of different arbitrary two marker points in the fourth coordinate set, multiple fourth distances are determined.

[0262] Preferably, the step can determine a reliable marker point distance set in the trusted space.

[0263] Illustratively, each two marker points form a set, and multiple sets of marker points with different spatial positions can be taken in the fourth coordinate set to correspond to the determination of multiple fourth distances.

[0264] S420, in the third coordinate set, a corresponding multiple third distances are determined.

[0265] Illustratively, the step can correspond to the determination of the reliable marker point distance set in S410 to determine a corresponding multiple third distances in the third coordinate set, i.e., to determine a third distance set corresponding to the reliable marker point distance set.

[0266] S430, using a statistical method, an average value of the ratio of the corresponding fourth distance to the third distance is determined, i.e., the second scale.

[0267] Illustratively, in the step, each distance in the reliable marker point distance set can be correspondingly compared with each distance in the third distance set to obtain multiple second scales; thereafter, an average value of the multiple second scales is obtained, i.e., an optimized second scale is obtained. That is, based on the corresponding multiple points in the third coordinate set and the fourth coordinate set, multiple second scales are determined; an average value of the multiple second scales is determined as the optimized second scale.

[0268] It should be noted that in the same flow of the three-dimensional scanning method, the first scale and the second scale can be optimized or one of them is optimized, and the embodiments of the present disclosure are not limited thereto.

[0269] The three-dimensional scanning method provided in the embodiment has at least the following beneficial effects:

[0270] First, scale-free photogrammetry: in the auxiliary measurement stage, the size measurement capability of the binocular vision measurement system at different working distances is used to estimate and optimize the scale of the results based on the photogrammetry principle, replacing the scale in the traditional photogrammetry scheme, simplifying the entire solution and reducing the cost.

[0271] Secondly, the dynamic photogrammetry function can display the three-dimensional reconstruction result of the mark point in real time in the photogrammetry stage, intuitively helping or guiding the user to judge the integrity of data acquisition, and the user does not need to collect a large number of redundant images to ensure the success rate, and the measurement result is seen and obtained, the robustness and precision are irrelevant to the user's experience, so as to improve the measurement efficiency, reliability, and reduce the learning cost of the user.

[0272] Thirdly, the photogrammetry without coding points, that is, the ordinary mark point space geometry coding technology is used instead of the special coding point in the photogrammetry stage and the three-dimensional scanning stage, so that the process of pasting the coding point on the workpiece is omitted, the process complexity is reduced, and the operation efficiency is improved.

[0273] It should be noted that, in this document, relational terms such as“first” and“second”, and the like, are used solely to distinguish one entity or action from another entity or action, without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms“comprises”,“comprising”, or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by“comprises a...” does not, without more constraints, preclude the existence of additional identical elements in the process, method, article, or apparatus that comprises the element.

[0274] The above description is merely one specific implementation of the present disclosure, making it possible for those skilled in the art to understand or implement the present disclosure. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to these embodiments described herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A three-dimensional scanning device, characterized in that, include: At least two image acquisition sensors are used to acquire images of the surface of the object under test; A binocular vision measurement calculator is used to reconstruct the three-dimensional surface features of the object to be measured in the image; A global error control calculator is used to calculate the surface features of the object under test in the image; A scale calculator is used to determine the scale based on the three-dimensional dataset output by the binocular vision measurement calculator and the global error control calculator in order to reconstruct the object to be measured. The binocular vision measurement calculator performs three-dimensional reconstruction of the surface features of the object to be measured in the image based on the principle of binocular vision, and obtains the corresponding three-dimensional dataset; the global error control calculator performs three-dimensional reconstruction of the surface features of the object to be measured in the image based on the principle of photogrammetry and / or the principle of mean statistical optimization, and obtains the corresponding three-dimensional dataset; the scale calculator replaces the physical scale with the scale determined by the three-dimensional dataset. The 3D scanning device also includes a marker encoder; The marker encoder is used to encode markers, and the encoded markers are used to compare and identify markers with the same name. The binocular vision measurement calculator is used to perform three-dimensional reconstruction of the marked points in the acquired image during the photogrammetry stage to determine the first coordinate set; and is used to perform three-dimensional reconstruction of the acquired image during the three-dimensional scanning stage to determine the fourth coordinate set and point cloud data. The global error control calculator is used to perform photogrammetric sparse bundle adjustment calculations on the marker points in the acquired image during the photogrammetric stage to obtain a second coordinate set. The scale calculator includes a scale estimation calculator and a scale optimization calculator; the scale includes a first scale and a second scale. The scale estimation calculator is used to determine the first scale based on the first coordinate set and the second coordinate set; Determine the third coordinate set based on the second coordinate set and the first scale; The scale optimization calculator is used to determine the second scale based on the fourth coordinate set and the third coordinate set; Determine the coordinate set of the marker points based on the third coordinate set and the second scale. The set of marker point coordinates is used to stitch together multiple frames of the point cloud data.

2. The three-dimensional scanning device according to claim 1, characterized in that: The image acquisition sensor is used to acquire an image of the surface of the object to be measured at a first working distance during the photogrammetry phase. The image acquisition sensor is also used to acquire images of the surface of the object to be measured at a second working distance during the three-dimensional scanning phase. Wherein, the second working distance is less than or equal to the first working distance.

3. The three-dimensional scanning device according to claim 2, characterized in that, It also includes pattern projectors; The pattern projector is used to project a structured light pattern onto the surface of the object to be measured during the three-dimensional scanning phase. The binocular vision measurement calculator is also used to perform three-dimensional reconstruction of the structured light pattern in the acquired image to obtain point cloud data.

4. The three-dimensional scanning device according to claim 1, characterized in that, The marker encoder is used to encode the marker points using a multi-point spatial structure encoding method.

5. A three-dimensional scanning method, characterized in that, The three-dimensional scanning method is performed using the three-dimensional scanning device according to any one of claims 1-4, and includes: At least two image acquisition sensors acquire images of the surface of the object under test; The binocular vision measurement calculator performs three-dimensional reconstruction of the surface features of the object to be measured in the image; The global error control calculator calculates the surface features of the object under test in the image; The scale calculator determines the scale based on the three-dimensional dataset output by the binocular vision measurement calculator and the global error control calculator in order to reconstruct the object to be measured. The binocular vision measurement calculator performs three-dimensional reconstruction of the surface features of the object to be measured in the image based on the principle of binocular vision, and obtains the corresponding three-dimensional dataset; the global error control calculator performs three-dimensional reconstruction of the surface features of the object to be measured in the image based on the principle of photogrammetry and / or the principle of mean statistical optimization, and obtains the corresponding three-dimensional dataset; the scale calculator replaces the physical scale with the scale determined by the three-dimensional dataset. The three-dimensional scanning device further includes a marker encoder, and the three-dimensional scanning method further includes: The marker encoder encodes the markers, and the encoded markers are used to compare and identify markers with the same name. The binocular vision measurement calculator performs three-dimensional reconstruction of the surface features of the object to be measured in the image, including: During the photogrammetry phase, the binocular vision measurement calculator performs three-dimensional reconstruction of the marker points in the acquired images to determine the first coordinate set; During the 3D scanning stage, the binocular vision measurement calculator performs 3D reconstruction on the acquired images to determine the fourth coordinate set and point cloud data; The global error control calculator calculates the surface features of the object to be tested in the image, including: During the photogrammetry phase, the global error control calculator performs photogrammetric sparse bundle adjustment on the marker points in the acquired image to obtain a second coordinate set; The scale calculator includes a scale estimation calculator and a scale optimization calculator; the scale includes a first scale and a second scale; the scale calculator determines the scale based on the 3D dataset output by the binocular vision measurement calculator and the global error control calculator to reconstruct the object to be measured, including: The scale estimation calculator is used to determine the first scale based on the first coordinate set and the second coordinate set; Determine the third coordinate set based on the second coordinate set and the first scale; The scale optimization calculator is used to determine the second scale based on the fourth coordinate set and the third coordinate set; Determine the coordinate set of the marker points based on the third coordinate set and the second scale. The set of marker point coordinates is used to stitch together multiple frames of the point cloud data.

6. The three-dimensional scanning method according to claim 5, characterized in that, The at least two image acquisition sensors acquire images of the surface of the object under test, including: During the photogrammetry phase, the image acquisition sensor acquires an image of the surface of the object to be measured at a first working distance; During the three-dimensional scanning phase, the image acquisition sensor acquires images of the surface of the object under test at a second working distance; Wherein, the second working distance is less than or equal to the first working distance.

7. The three-dimensional scanning method according to claim 5, characterized in that, At least some of the coordinate points in the fourth coordinate set are in the trusted space, and the second scale is determined using the coordinate points in the fourth coordinate set that are located in the trusted space. The trusted space is a spatial frustum within the measurement space of the 3D scanning device. The marker points within the spatial frustum satisfy the following: the alignment error, back projection error, and photogrammetric accuracy of the marker points are on the same order of magnitude according to a preset statistical ratio.

8. The three-dimensional scanning method according to claim 6, characterized in that, The three-dimensional scanning device further includes a pattern projector, and the three-dimensional scanning method further includes: During the three-dimensional scanning phase, the pattern projector projects a structured light pattern onto the surface of the object to be measured; The binocular vision measurement calculator performs three-dimensional reconstruction of the structured light pattern in the acquired image to obtain point cloud data.

9. The three-dimensional scanning method according to claim 5, characterized in that, The marker encoder uses a multi-point spatial structure encoding method to encode the marker points.

Images