Three-dimensional measurement method and apparatus, and storage medium and electronic apparatus

Abstract

A three-dimensional measurement method and apparatus, and a storage medium and an electronic apparatus. The method comprises: collecting images of a measured object from different viewing angles by means of a plurality of cameras, wherein a grating pattern image is projected onto the measured object; on the basis of intrinsic and extrinsic parameters of each camera after calibration and the collected images of the measured object, determining point cloud data corresponding to each camera; and aligning the point cloud data corresponding to each camera to the same coordinate system, so as to obtain point cloud data of a target coordinate system, and performing three-dimensional measurement on the basis of the point cloud data of the target coordinate system.

Description

Three-dimensional measurement method, device, storage medium and electronic device

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This disclosure is based on Chinese patent application CN202311252715.1 filed on September 26, 2023, entitled “Three-dimensional measurement method, device, storage medium and electronic device”, and claims the priority of the patent application, and all the contents disclosed therein are incorporated into this disclosure by reference. Technical Field

[0003] The embodiments of the present disclosure relate to the field of machine vision technology, and more specifically, to a three-dimensional measurement method, device, storage medium, and electronic device. Background Art

[0004] Three-dimensional (3D) high-precision measurement uses high-precision instruments and techniques to determine the geometric shape and dimensions of an object's surface in three dimensions. This method is used in manufacturing, construction, medicine, and other fields requiring precise measurement and inspection.

[0005] Common 3D machine vision measurement methods include laser measurement, monocular structured light measurement, and binocular measurement. These three measurement methods can only solve measurement problems in certain scenarios and cannot meet the requirements of sub-micron accuracy.

[0006] Summary of the Invention

[0007] The embodiments of the present disclosure provide a three-dimensional measurement method, device, storage medium, and electronic device to at least solve the problem in the related art that a three-dimensional measurement method that can meet sub-micron precision requirements is needed.

[0008] According to one embodiment of the present disclosure, a three-dimensional measurement method is provided, comprising: acquiring images of a measured object from different perspectives using multiple cameras, wherein a grating stripe image is projected onto the measured object; determining point cloud data corresponding to each camera based on calibrated internal and external parameters of each camera and the acquired image of the measured object; aligning the point cloud data corresponding to each camera into the same coordinate system to obtain point cloud data of a target coordinate system, and performing three-dimensional measurement based on the point cloud data of the target coordinate system.

[0009] According to another embodiment of the present disclosure, a three-dimensional measurement device is provided, including: an acquisition module, configured to acquire images of a measured object from different perspectives using multiple cameras, wherein a sinusoidal stripe of structured light is projected onto the measured object; a determination module, configured to determine point cloud data corresponding to each camera based on calibrated internal and external parameters of each camera and the acquired image of the measured object; and a measurement module, configured to align the point cloud data corresponding to each camera into the same coordinate system to obtain point cloud data of a target coordinate system, and perform three-dimensional measurement based on the point cloud data of the target coordinate system.

[0010] According to another embodiment of the present disclosure, a computer-readable storage medium is provided, in which a computer program is stored. The computer program is configured to execute the steps of any one of the above method embodiments when running.

[0011] According to another embodiment of the present disclosure, an electronic device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to perform the steps in any one of the above method embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] FIG1 is a hardware structure block diagram of a computer terminal for running a three-dimensional measurement method according to an embodiment of the present disclosure;

[0013] FIG2 is a flow chart of a three-dimensional measurement method according to an embodiment of the present disclosure;

[0014] FIG3 is a structural block diagram of a three-dimensional measurement device according to an embodiment of the present disclosure;

[0015] FIG4 is a structural block diagram of a 3D high-precision measurement system according to an embodiment of the present disclosure;

[0016] FIG5 is a flow chart of a 3D high-precision measurement method based on four-eye structured light according to an embodiment of the present disclosure;

[0017] FIG6 is a waveform diagram of sinusoidal light with different phase shifts according to an embodiment of the present disclosure;

[0018] FIG7 is a waveform diagram of a phase main value according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0019] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings and in conjunction with embodiments.

[0020] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0021] The method embodiments provided in the embodiments of the present disclosure can be executed in a mobile terminal, a computer terminal or a similar computing device. Taking running on a computer terminal as an example, FIG1 is a hardware structure block diagram of a computer terminal running a three-dimensional measurement method in an embodiment of the present disclosure. As shown in FIG1 , the computer terminal may include one or more (only one is shown in FIG1 ) processors 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data, wherein the above-mentioned computer terminal may also include a transmission device 106 and an input and output device 108 for communication functions. It can be understood by those skilled in the art that the structure shown in FIG1 is only for illustration and does not limit the structure of the above-mentioned computer terminal. For example, the computer terminal may also include more or fewer components than those shown in FIG1 , or have a configuration different from that shown in FIG1 .

[0022] The memory 104 can be used to store computer programs, such as software programs and modules of application software, such as the computer program corresponding to the three-dimensional measurement method in the embodiment of the present disclosure. The processor 102 executes the computer program stored in the memory 104 to perform various functional applications and data processing, thereby implementing the above-mentioned method. The memory 104 may include high-speed random access memory and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories may be connected to the computer terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0023] The transmission device 106 is used to receive or transmit data via a network. A specific example of the aforementioned network may include a wireless network provided by a computer terminal's communications provider. In one embodiment, the transmission device 106 includes a network interface controller (NIC), which can be connected to other network devices via a base station to enable communication with the Internet. In another embodiment, the transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0024] In this embodiment, a three-dimensional measurement method running on the above-mentioned computer terminal is provided. FIG2 is a flow chart of the three-dimensional measurement method according to an embodiment of the present disclosure. As shown in FIG2 , the flow chart includes the following steps:

[0025] Step S202, capturing images of the object under test at different viewing angles using multiple cameras, wherein a grating fringe image is projected onto the object under test;

[0026] In one embodiment, the plurality of cameras includes more than two cameras.

[0027] In one embodiment, the plurality of cameras are four cameras, and the four cameras are spaced the same distance apart and have the same distance from the object to be measured.

[0028] In one embodiment, four high-resolution industrial cameras (e.g., CCDs) are deployed in the middle of each of the four sides of a square, and a DMD (Digital Micromirror Device) structured light projector is deployed in the center of the square to image workpieces within a common field of view. For example, a grating fringe image can be projected onto the object being measured.

[0029] By combining four industrial cameras and structured light, high-quality original image data can be obtained, and multiple perspectives can avoid problems such as shadows, occlusions, and missing images.

[0030] Step S204, determining the point cloud data corresponding to each camera based on the calibrated internal and external parameters of each camera and the collected image of the measured object;

[0031] Before step S204 of this embodiment, the method further includes: calibrating the internal parameters and external parameters of each camera using a calibration plate, so that each camera corrects the distortion in the captured image based on the calibrated internal parameters and external parameters, wherein the calibration plate is set at the center of the area enclosed by the multiple cameras.

[0032] In one embodiment, a calibration plate with a white background and a blue circular pattern is used. By measuring the diameter or circumference of the circle, the scale of the image can be accurately calculated and distortion corrected, improving the geometric accuracy of the image. Compared to calibration plates of other shapes, circular calibration plates are more tolerant to changes in lighting conditions and viewing angles, providing more stable and reliable calibration results.

[0033] In step S204 of this embodiment, the point cloud data corresponding to each camera is determined, including: according to the calibrated internal and external parameters of each camera and the collected image of the object to be measured, and the point cloud data corresponding to each camera is determined by a multi-step phase method and a parallax calculation method.

[0034] In one embodiment, point cloud data corresponding to each camera is determined by a multi-step phase method and a parallax calculation method, including: for any camera among the multiple cameras, obtaining multiple images of the object under test captured by the camera, wherein the phases of the grating fringe images projected on the object under test in the multiple images of the object under test are different, and wherein the grating fringe images are sinusoidal grating fringe images; obtaining a relationship model between phase and three-dimensional spatial coordinates based on the grating fringe images with phase differences in the multiple images of the object under test; and generating point cloud data corresponding to the camera based on the two-dimensional coordinates of each pixel point in the multiple images of the object under test, the relationship model between the phase and the three-dimensional spatial coordinates, and the internal and external parameters of the camera.

[0035] By calibrating the intrinsic and extrinsic parameters of the camera, the mapping relationship between the camera coordinate system and the world coordinate system is established, and the pixel coordinates of the camera image are obtained for subsequent point cloud data generation.

[0036] Step S206 , aligning the point cloud data corresponding to each camera into the same coordinate system to obtain point cloud data of a target coordinate system, and performing three-dimensional measurement based on the point cloud data of the target coordinate system.

[0037] Before step S206 of this embodiment, the method further includes: performing at least one of the following data processing on the point cloud data: cleaning, denoising, filtering, and segmentation.

[0038] In one embodiment, the filtering processing of point cloud data mainly includes through-filtering and statistical filtering. The function of through-filtering is to retain the part of the collected point cloud data required for measurement, and filter out the rest through the value range of the three-dimensional space coordinates, while the statistical filtering is to filter out the noise data between the point clouds.

[0039] In step S206 of this embodiment, the process includes: iteratively finding the minimum distance between point cloud data of any two cameras to match the point cloud data, wherein, for the point cloud data of any two cameras, the point cloud data of one camera is regarded as reference point cloud data, and the point cloud data of the other camera is regarded as point cloud data to be registered; determining a transformation matrix based on the matched point cloud data, and performing coordinate transformation on the point cloud data to be registered according to the transformation matrix to convert the point cloud data to be registered from an original coordinate system to a target coordinate system.

[0040] In one embodiment, through operations such as rotation and translation and based on ICP (Iterative Closest Point, iterative closest point algorithm), the minimum distance between the two sets of point cloud data is iteratively found, and a transformation matrix is ​​obtained. The coordinate transformation of the point cloud data to be registered is performed according to the transformation matrix, and it is converted from the original coordinate system to the target coordinate system, that is, the coordinate system corresponding to the reference point cloud data. At the same time, the distance-based deduplication method regards the points with a close distance in the point cloud data to be registered as the same point, and only retains one of the points to remove redundant points, thereby reducing the amount of data and improving the efficiency of point cloud processing. The point cloud data obtained after registration and the remaining unregistered point cloud data are point cloud registered according to the above-mentioned point cloud registration method until all point cloud data are aligned to the same coordinate system.

[0041] Through the above steps, the combination of multiple cameras and structured light can avoid the effects of shadows, occlusions, and omissions on image data quality, resulting in high-quality raw image data. Based on the high-quality image data from multiple perspectives, the point cloud data corresponding to each camera is determined, and the point cloud data corresponding to each camera is aligned to the same coordinate system for fusion of the three-dimensional point cloud data, resulting in a three-dimensional measurement result that meets submicron requirements. This solves the problem of the related art requiring a three-dimensional measurement method that can meet submicron precision requirements, thereby achieving improved three-dimensional measurement accuracy.

[0042] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus the necessary general hardware platform, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present disclosure is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), including a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present disclosure.

[0043] This embodiment also provides a three-dimensional measurement device for implementing the above-mentioned embodiments and preferred implementations. Details already described will not be repeated. As used below, the term "module" may refer to a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented using software, implementation using hardware, or a combination of software and hardware, is also possible and contemplated.

[0044] FIG3 is a structural block diagram of a three-dimensional measurement device according to an embodiment of the present disclosure. As shown in FIG3 , the device includes: an acquisition module 31 , a determination module 32 , and a measurement module 33 .

[0045] An acquisition module 31 is configured to acquire images of the object under test at different viewing angles using multiple cameras, wherein a grating fringe image is projected onto the object under test;

[0046] In one embodiment, the plurality of cameras includes more than two cameras.

[0047] a determination module 32 configured to determine point cloud data corresponding to each camera based on the calibrated internal and external parameters of each camera and the collected image of the measured object;

[0048] The measurement module 33 is configured to align the point cloud data corresponding to each camera into the same coordinate system, obtain point cloud data of a target coordinate system, and perform three-dimensional measurement based on the point cloud data of the target coordinate system.

[0049] It should be noted that the above modules can be implemented through software or hardware. For the latter, it can be implemented in the following ways, but not limited to: the above modules are all located in the same processor; or the above modules are located in different processors in any combination.

[0050] To facilitate understanding of the technical solutions provided by the present disclosure, embodiments of the present disclosure will be described in detail below with reference to specific scenarios.

[0051] Common 3D machine vision measurement methods include laser measurement, monocular structured light measurement, and binocular measurement. The following is an explanation of three common 3D machine vision measurement methods and the shortcomings of each method.

[0052] Laser measurement can achieve micron-level accuracy and is unaffected by the color and texture of the object being measured. It is suitable for a wide range of measurement tasks and can acquire large amounts of data in a short period of time. However, laser measurement is sensitive to ambient light and requires a relatively stable environment. It also has high equipment costs and strict site requirements.

[0053] Monocular structured light measurement enables fast, non-contact measurement with millimeter to submillimeter accuracy, making it suitable for 3D measurement of small and medium-sized objects. Furthermore, the structured light measurement equipment used is relatively inexpensive, making it easy to apply. However, monocular structured light measurement is susceptible to the surface material and reflectivity of the object being measured, requiring parameter adjustments for different objects.

[0054] Binocular measurement can acquire image data from different perspectives to obtain relatively comprehensive information, and is suitable for three-dimensional measurement of various objects. Its accuracy range is at the millimeter level, but it requires precise calibration and registration, and its accuracy is affected by factors such as ambient light and noise, and it requires a relatively long measurement time.

[0055] In addition to their own shortcomings, the above three measurement methods are unable to simultaneously meet the requirements of submicron accuracy and low-cost competitiveness.

[0056] To simultaneously meet sub-micron precision requirements and low-cost competitiveness, we can consider it from the perspective of data acquisition end design, use common standard components to build a high-resolution, multi-perspective acquisition end, and obtain high-quality, dense, and complete point cloud data for application function development, achieving sub-micron and low-cost requirements.

[0057] The disclosed embodiments propose a 3D high-precision measurement method based on 4-eye structured light. 4-eye high-speed and high-resolution CCD image sensors and lenses are used to image the reflected light from the projected area array stripe structured light from multiple directions. This method can solve the common problems of unilateral shadows and point cloud missing in the operation of traditional monocular structured light 3D cameras / line laser 3D cameras, output a denser and more complete 3D point cloud image, and then achieve high-precision measurement of related positions by preprocessing the point cloud data, plane fitting, and distance calculation, thereby meeting the technical requirements of high-precision 3D visual inspection at the sub-micron level and at low cost.

[0058] The disclosed embodiments can be applied to the quality inspection of components in the 3C electronics industry. In addition, they can also be applied to the manufacture and testing of aerospace parts and equipment, the manufacture and quality control of automobile parts, various experiments and measurements in scientific research, etc.

[0059] This high-precision 3D measurement method based on four-eye structured light can be applied to high-precision measurement fields such as stereoscopic dimension measurement, precise spatial positioning guidance, and device surface shape analysis in the 3C electronics manufacturing industry. It is suitable for measuring workpieces with a depth of field of less than 70mm and a field of view of 75x30mm. The disclosed embodiments have certain requirements for hardware computing power. Higher computing power reduces runtime, enabling high-speed operation.

[0060] FIG4 is a structural block diagram of a 3D high-precision measurement system according to an embodiment of the present disclosure. As shown in FIG4 , the system mainly includes five modules: hardware construction, parameter calibration, point cloud construction, point cloud processing, and point cloud registration.

[0061] Hardware construction, which involves selecting and assembling cameras, structured light projectors, industrial computers, data cables, acquisition cards, calibration boards, and other equipment, as well as performing calibration and debugging, is the foundation for 3D visual measurement. This hardware system enables the acquisition of high-quality image data, enabling precise 3D measurement. Proper selection and optimization can improve measurement accuracy and efficiency, while also providing customized solutions for different application scenarios.

[0062] Calibration is used to determine the internal and external parameters of devices such as cameras and laser scanners for accurate 3D measurement. This process typically involves two aspects: camera internal calibration and external calibration. Camera internal calibration primarily determines parameters such as focal length and distortion, which affect the accuracy and quality of camera imaging. External calibration determines parameters such as the camera's position and attitude. Correct calibration eliminates inherent device errors, enabling more accurate 3D measurement.

[0063] Point cloud construction is set to convert discrete point data of a three-dimensional scene or object surface into a point cloud model, scan and capture object surface image information through industrial cameras and structured light, and generate point cloud data using a phase shift method.

[0064] Point cloud processing is the process of processing and analyzing collected point cloud data, extracting useful information and converting it into visualizations or digital models. Point cloud processing includes functions such as data representation, statistical filtering, Euclidean clustering, MLS, and point cloud downsampling.

[0065] Point cloud registration is the process of aligning multiple point cloud data into the same coordinate system for subsequent processing and analysis, including the selection of reference point clouds, feature matching, calculation of transformation matrices, accuracy assessment and other functions.

[0066] FIG5 is a flow chart of a 3D high-precision measurement method based on four-eye structured light according to an embodiment of the present disclosure. As shown in FIG5 , the method includes the following steps:

[0067] S501, build the hardware environment, that is, use the DMD module to project coded structured light and four high-resolution industrial cameras to collect image data;

[0068] For example, four high-resolution industrial cameras are deployed in the middle of the four sides of a square, and a DMD structured light projector is deployed in the middle of the square to image the workpiece within the public field of view.

[0069] S502, calibrate the camera internal parameters and system parameters with the aid of a calibration plate, and then project the grating fringe pattern onto the surface of the object to be measured to obtain the relationship between the phase and the three-dimensional space coordinates.

[0070] In this embodiment, a circular-patterned calibration plate is used to calibrate the internal parameters and system parameters of four high-resolution industrial cameras, and a mapping relationship between the camera coordinate system and the world coordinate system is established to obtain the camera image pixel coordinates; sinusoidal stripes are generated by computer programming, and the sinusoidal stripes are projected onto the object to be measured through a projection device, and a CCD (i.e., industrial camera) camera is used to capture the degree of bending of the stripes modulated by the object, and the curved stripes are demodulated to obtain the phase, which is then converted into spatial height information, so as to obtain a relationship model between the desired question and the three-dimensional spatial coordinates.

[0071] In one embodiment, a calibration plate with a white background and a blue circular pattern was designed to comprehensively evaluate the accuracy of pattern recognition and positioning, using surface reflections and pattern recognition. By measuring the diameter or circumference of the circle, the scale of the image can be accurately calculated and distortions such as radial and tangential distortion can be corrected, improving the geometric accuracy of the image. By photographing the calibration plate from different perspectives (i.e., four camera views) and analyzing the pixel coordinates of the markers, the camera's position and posture can be calculated, enabling 3D reconstruction and depth estimation.

[0072] Compared with calibration plates of other shapes, circular calibration plates are more tolerant to changes in lighting conditions and viewing angles, and can provide more stable and reliable calibration results.

[0073] The projection module generates a sinusoidal grating fringe pattern with a certain phase shift through computer programming and projects this sinusoidal grating fringe pattern onto the object being measured. The camera then simultaneously captures a set of grating images. By analyzing this set of grating images with a certain phase difference, the initial phase value of the surface topography data of the object being measured is calculated. Phase unwrapping is performed using the correct initial phase value and more coding patterns to obtain an unambiguous absolute phase value within the image frame. The light intensity with a phase shift can usually be expressed as:

[0074] In(x, y) = A(x, y) + B(x, y)cos [Φ(x, y) + an] Formula (1)

[0075] In this embodiment, a four-step phase shift method is adopted, and the phase shift steps are: 0, π / 2, π, 3π / 2, and the light intensity expressions are:

[0076] In(x, y) = A(x, y) + B(x, y)cos [Φ(x, y)]; Formula (2)

[0077] In(x, y) = A(x, y) + B(x, y)cos[Φ(x, y) +π / 2]; Formula (3)

[0078] In(x, y) = A(x, y) + B(x, y)cos [Φ(x, y) + π]; Formula (4)

[0079] In(x, y) = A(x, y) + B(x, y)cos[Φ(x, y) +3π / 2]; Formula (5)

[0080] By performing phase extraction on four grating images with different phase shifts, the following formula (6) can be used to calculate the main phase value.

[0081] Φ(x, y) = arctan( (I2 - I4) / (I1 - I3) ) Formula (6)

[0082] Four sinusoidal light intensities with different phase shifts (i.e., (1) to (4) in Figure 6) are displayed in the same coordinate system. As shown in Figure 6, by solving the image phase principal value based on the four-step phase shift method, a phase principal value diagram can be obtained. As shown in Figure 7, the phase principal value varies periodically in the image. However, since the phase principal value obtained by a single frequency cannot completely cover the pixel distribution of the entire image, this may result in pixels at different positions having the same phase principal value. Therefore, the phase principal value needs to be expanded. By expanding the periodically varying phase principal value, the absolute phase that can cover the entire image is obtained.

[0083] S503, point cloud data generation, obtaining point cloud data from each camera perspective through multi-step phase method and parallax calculation;

[0084] Through the relationship model between the camera image pixel coordinates and phase and the three-dimensional space coordinates obtained in the calibration process, using the known pixel coordinates (m, n, θ) and a1-a8 (i.e., the intrinsic and extrinsic parameters of the camera), through the simultaneous model formula and the camera internal and external parameter formula, we can obtain the relationship between Xc, Yc, and Zc with respect to θ (the brightness value of the pixel point) and the intrinsic parameter, thereby calculating the three-dimensional space coordinates of the corresponding pixel and generating point cloud data.

[0085] S504, point cloud preprocessing, performing at least one of the following operations on the original point cloud data (i.e., the point cloud data obtained in step S503): cleaning, denoising, filtering, downsampling, and segmentation.

[0086] Point cloud data processing mainly includes straight-through filtering and statistical filtering. The function of straight-through filtering is to retain the part of the collected point cloud required for measurement, and filter out the rest through the value range of three-dimensional space coordinates (X, Y, Z). Statistical filtering filters out outliers (i.e., noise data) between point clouds, downsamples the point cloud data, and adjusts the point cloud density.

[0087] In one embodiment, a statistical filtering algorithm performs a statistical analysis of the neighborhood of each point, filtering out outliers that do not meet the requirements based on the distribution characteristics of the distances from the target point to all its neighboring points. The algorithm iterates the entire input twice: in the first iteration, the average distance from each point to its k nearest neighbors is calculated, and the result is Gaussian. The mean u and standard deviation σ of all these average distances are then calculated to determine the distance threshold thresh_d, where thresh_d = u ± kσ, where k is the standard deviation multiplier. In the second iteration, if the average neighborhood distance of these points is lower or higher than the threshold, respectively, these points are classified as inliers or outliers.

[0088] S505, point cloud registration, completes the precise registration of the four sets of point cloud data through operations such as rotation, translation, and deduplication.

[0089] In one embodiment, for any two sets of point cloud data, one set of point cloud data is used as the reference point cloud data, and the other set of point cloud data is used as the point cloud data to be registered. Based on the ICP (Iterative Closest Point) algorithm, the minimum distance between the two sets of point cloud data is iteratively found through operations such as rotation and translation, and a transformation matrix is ​​obtained. The coordinates of the point cloud data to be registered are transformed according to the transformation matrix, and it is converted from the original coordinate system to the target coordinate system (i.e., the coordinate system corresponding to the reference point cloud data). At the same time, the distance-based deduplication method treats points with close distances in the point cloud data to be registered as the same point, retaining only one of the points to remove redundant points, thereby reducing the amount of data and improving the efficiency of point cloud processing.

[0090] The point cloud data obtained after registration and the remaining unregistered point cloud data are registered according to the above point cloud registration method until all the point cloud data are aligned to the same coordinate system.

[0091] S506, measurement application, extracts key features to realize application functions such as height distance calculation and coplanarity measurement.

[0092] In one embodiment, the parameters of a data model containing noise and outliers are estimated through an iterative algorithm based on random sampling using RANSAC, thereby fitting the corresponding plane equation. The obtained plane equation is then used to perform spacing calculations to obtain the distance difference between the planes, which is then applied to various measurement calculations.

[0093] Among them, for the plane calculation process, it is first necessary to select the point cloud data through ROI (Region of Interest) to obtain the point cloud data to be fitted; then use the plane equation in space ax+by+cz+d=0 as the model equation, and use the fitting algorithm to fit it to obtain the corresponding plane equation.

[0094] The principle of RANSAC algorithm is as follows:

[0095] Step S1, selecting the minimum data sample for the estimation model, for example: for two-dimensional or three-dimensional straight line fitting, at least two points are required to determine a straight line; for three-dimensional plane fitting, at least three points are required to determine a three-dimensional plane.

[0096] Step S2: Using the minimum data sample, calculate the fitting model, that is, the equation of the line or the equation of the plane.

[0097] In step S3, all models are brought into the fitted model to calculate the number of inliers, that is, the number of data points whose errors between the data points and the fitted model are within a certain threshold range.

[0098] Step S4, comparing the number of inliers of the current model and the best model obtained in the previous iteration, recording the model parameters and the number of inliers of the model with the largest number of inliers, wherein the more inliers there are, the better the model.

[0099] Repeat steps S1-S4 until the iteration termination condition is reached, for example, the maximum number of iterations is reached or the number of inliers reaches the iteration termination condition.

[0100] Finally, after obtaining the plane equations of each plane, the height of each plane can be calculated. In the process of calculating the coplanarity, considering the impact of data fluctuations and noise on the results, in this embodiment, 10% of the highest points and 10% of the lowest points are filtered out, and then the closest distance is subtracted from the farthest distance.

[0101] Through the above-mentioned embodiments of the present disclosure, four industrial cameras are combined with structured light to obtain high-quality original image data, avoiding shadows, occlusions, missing phenomena, etc.; multi-view 3D point cloud fusion technology based on multi-view stereo imaging technology further improves the measurement repeatability accuracy and reaches sub-micron requirements; universal basic standard components are used, and through solution design, the expected effect is achieved, costs are reduced, and product competitiveness is improved.

[0102] The technical solutions in the embodiments of the present disclosure can be replicated in other scenarios such as 3C electronics, semiconductors, and precision machining, and can effectively improve the accuracy and reliability of the detection process. At the same time, the non-contact measurement method can also improve measurement efficiency and the possibility of automated production.

[0103] An embodiment of the present disclosure further provides a computer-readable storage medium, in which a computer program is stored. The computer program is configured to execute the steps of any one of the above method embodiments when run.

[0104] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk, or an optical disk.

[0105] An embodiment of the present disclosure further provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to perform the steps in any of the above method embodiments.

[0106] In an exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0107] For specific examples in this embodiment, reference may be made to the examples described in the above embodiments and exemplary implementation modes, and this embodiment will not be described in detail here.

[0108] Obviously, those skilled in the art should understand that the modules or steps of the present disclosure described above can be implemented using a general-purpose computing device, they can be concentrated on a single computing device, or distributed across a network composed of multiple computing devices, they can be implemented using program code executable by the computing device, and thus, they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be performed in a different order than herein, or they can be fabricated into separate integrated circuit modules, or multiple modules or steps can be fabricated into a single integrated circuit module for implementation. Thus, the present disclosure is not limited to any particular combination of hardware and software.

[0109] The foregoing description is merely a preferred embodiment of the present disclosure and is not intended to limit the present disclosure. Those skilled in the art will readily appreciate that various modifications and variations of the present disclosure are possible. Any modifications, equivalent substitutions, or improvements made within the principles of the present disclosure shall be included within the scope of protection of the present disclosure.

Claims

1. A three-dimensional measurement method, comprising: The images of the object to be measured are collected at different viewing angles by using multiple cameras, wherein a grating fringe image is projected onto the object to be measured; Determine the point cloud data corresponding to each camera according to the calibrated internal and external parameters of each camera and the collected image of the object to be measured; The point cloud data corresponding to each camera is aligned to the same coordinate system to obtain the point cloud data of the target coordinate system, and three-dimensional measurement is performed based on the point cloud data of the target coordinate system.

2. The method according to claim 1, wherein: The multiple cameras are four cameras, and the four cameras have the same spacing between them and the same distance from the measured object.

3. The method according to claim 1, wherein: According to the calibrated internal and external parameters of each camera and the collected image of the measured object, the method further includes: The internal parameters and external parameters of each camera are calibrated by a calibration plate, so that each camera corrects the distortion in the captured image based on the calibrated internal parameters and external parameters, wherein the calibration plate is arranged at the center of the area surrounded by the multiple cameras.

4. The method according to claim 1, wherein: Determine the point cloud data corresponding to each camera, including: According to the calibrated internal and external parameters of each camera and the collected image of the object to be measured, the point cloud data corresponding to each camera is determined by a multi-step phase method and a parallax calculation method.

5. The method according to claim 4, wherein: The point cloud data corresponding to each camera is determined by the multi-step phase method and parallax calculation method, including: For any camera among the multiple cameras, a plurality of images of the object under test captured by the camera are acquired, wherein the phases of the grating fringe images projected on the object under test in the multiple images of the object under test are different, and the grating fringe images are sinusoidal grating fringe images; Acquire a relationship model between phase and three-dimensional space coordinates according to the grating fringe images with phase difference in the plurality of images of the measured object; Point cloud data corresponding to the camera direction is generated according to the two-dimensional coordinates of each pixel point in the multiple images of the measured object, the relationship model between the phase and the three-dimensional space coordinates, and the internal and external parameters of the camera.

6. The method according to claim 1, wherein: Before aligning the point cloud data corresponding to each camera into the same coordinate system, the method further includes: The point cloud data is subjected to at least one of the following data processing steps: cleaning, denoising, filtering, and segmentation.

7. The method according to claim 1, wherein: Aligning the point cloud data corresponding to each camera into the same coordinate system includes: Matching the point cloud data is performed by iteratively finding the minimum distance between the point cloud data of any two cameras, wherein, for the point cloud data of any two cameras, the point cloud data of one camera is regarded as the reference point cloud data, and the other is regarded as the point cloud data to be registered; A transformation matrix is ​​determined based on the matched point cloud data, and coordinate transformation is performed on the point cloud data to be registered according to the transformation matrix, so as to convert the point cloud data to be registered from an original coordinate system to a target coordinate system.

8. A three-dimensional measuring device, comprising: An acquisition module is configured to acquire images of the object under test at different viewing angles through multiple cameras, wherein a sinusoidal-fringe structured light is projected onto the object under test; A determination module is configured to determine the point cloud data corresponding to each camera according to the calibrated internal and external parameters of each camera and the collected image of the measured object; The measurement module is configured to align the point cloud data corresponding to each camera into the same coordinate system, obtain the point cloud data of the target coordinate system, and perform three-dimensional measurement based on the point cloud data of the target coordinate system.

9. A computer-readable storage medium having a computer program stored therein, wherein: When the computer program is executed by a processor, the steps of the method described in any one of claims 1 to 7 are implemented.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method described in any one of claims 1 to 7 when executing the computer program.

Images