Three-dimensional measurement method and device, storage medium and electronic device

Abstract

The embodiment of the invention provides a three-dimensional measurement method and device, a storage medium and an electronic device. The method comprises the steps that images of a measured object at different visual angles are collected through a plurality of cameras, and a grating stripe image is projected on the measured object; determining point cloud data corresponding to each camera according to the calibrated internal and external parameters of each camera and the acquired image of the measured object; and aligning the point cloud data corresponding to each camera to the same coordinate system to obtain point cloud data of a target coordinate system, and performing three-dimensional measurement based on the point cloud data of the target coordinate system. According to the method and the device, the problem that a three-dimensional measurement method capable of meeting the submicron precision requirement is needed in the prior art is solved, and the effect of improving the three-dimensional measurement precision is achieved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of machine vision technology, and more specifically, to a three-dimensional measurement method, device, storage medium, and electronic device. Background Art

[0002] Three-dimensional (3D) high-precision measurement refers to a method that uses high-precision instruments and technologies to obtain the surface geometry and size of objects in three-dimensional space. This method can be used in manufacturing, construction, medicine and other fields that require precise measurement and inspection.

[0003] Common 3D machine vision measurement methods include laser measurement, monocular structured light measurement, and binocular measurement. These three measurement methods can only solve measurement problems in some scenarios and cannot meet the requirements of sub-micron accuracy. Summary of the invention

[0004] The embodiments of the present application provide a three-dimensional measurement method, device, storage medium and electronic device to at least solve the problem in the related art that a three-dimensional measurement method that can meet sub-micron accuracy requirements is needed.

[0005] According to one embodiment of the present application, a three-dimensional measurement method is provided, comprising: acquiring images of a measured object at different viewing angles through multiple cameras, wherein a grating stripe image is projected onto the measured object; determining point cloud data corresponding to each camera according to calibrated internal and external parameters of each camera and the acquired image of the measured object; aligning the point cloud data corresponding to each camera to the same coordinate system to obtain point cloud data of a target coordinate system, and performing three-dimensional measurement based on the point cloud data of the target coordinate system.

[0006] According to another embodiment of the present application, a three-dimensional measurement device is provided, including: an acquisition module, used to acquire images of a measured object at different viewing angles through multiple cameras, wherein a structured light with sinusoidal stripes is projected onto the measured object; a determination module, used to determine point cloud data corresponding to each camera based on the calibrated internal and external parameters of each camera and the acquired image of the measured object; and a measurement module, used to align the point cloud data corresponding to each camera to the same coordinate system to obtain point cloud data of a target coordinate system, and perform three-dimensional measurement based on the point cloud data of the target coordinate system.

[0007] According to another embodiment of the present application, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above method embodiments when run.

[0008] According to another embodiment of the present application, an electronic device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0009] Through the above-mentioned embodiments of the present application, due to the combination of multiple cameras and structured light, the effects of shadows, occlusions, missing, etc. on the quality of image data can be avoided, and high-quality original image data can be obtained; based on the high-quality image data under multiple perspectives, the point cloud data corresponding to each camera is determined, and the point cloud data corresponding to each camera is aligned to the same coordinate system to perform fusion of three-dimensional point cloud data, and a three-dimensional measurement result that meets the sub-micron level requirements can be obtained. Therefore, the problem of the need for a three-dimensional measurement method that can meet the sub-micron level accuracy requirements in the related art can be solved, and the effect of improving the three-dimensional measurement accuracy can be achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 is a hardware structure block diagram of a computer terminal for running a three-dimensional measurement method according to an embodiment of the present application;

[0011] Figure 2 is a flow chart of a three-dimensional measurement method according to an embodiment of the present application;

[0012] Figure 3 is a structural block diagram of a three-dimensional measurement device according to an embodiment of the present application;

[0013] Figure 4 is a structural block diagram of a 3D high-precision measurement system according to an embodiment of the present application;

[0014] Figure 5 is a flow chart of a 3D high-precision measurement method based on 4-eye structured light according to an embodiment of the present application;

[0015] Figure 6 is a waveform diagram of sinusoidal light with different phase shifts according to an embodiment of the present application;

[0016] Figure 7 is a waveform diagram of the phase main value according to an embodiment of the present application. DETAILED DESCRIPTION

[0017] The embodiments of the present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0018] It should be noted that the terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0019] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Taking running on a computer terminal as an example, Figure 1 is a hardware structure block diagram of a computer terminal for running a three-dimensional measurement method according to an embodiment of the present application. Figure 1 As shown, the computer terminal may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data, wherein the above-mentioned computer terminal may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above-mentioned computer terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown.

[0020] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the three-dimensional measurement method in the embodiment of the present application. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories can be connected to the computer terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0021] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of a computer terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0022] In this embodiment, a three-dimensional measurement method running on the above-mentioned computer terminal is provided. Figure 2 is a flow chart of a three-dimensional measurement method according to an embodiment of the present application, such as Figure 2As shown, the process includes the following steps:

[0023] Step S202, collecting images of the object under test at different viewing angles by using multiple cameras, wherein a grating fringe image is projected onto the object under test;

[0024] In one embodiment, the plurality of cameras includes more than two cameras.

[0025] In one embodiment, the multiple cameras are four cameras, and the four cameras have the same spacing between them and the same distance from the measured object.

[0026] In one embodiment, four high-resolution industrial cameras (such as CCD) are respectively deployed in the middle of the four sides of the square in the form of a square, and a DMD (Digital Micromirror Device) structured light projector is deployed in the middle of the square to image the workpiece within the public field of view, for example, projecting a grating fringe image on the object to be measured.

[0027] By combining 4 industrial cameras and structured light, high-quality original image data can be obtained, and multiple perspectives can avoid problems such as shadows, occlusions, and missing images.

[0028] Step S204, determining the point cloud data corresponding to each camera according to the calibrated internal and external parameters of each camera and the collected image of the measured object;

[0029] Before step S204 of this embodiment, the method further includes: calibrating the internal parameters and external parameters of each camera through a calibration plate, so that each camera corrects the distortion in the captured image based on the calibrated internal parameters and external parameters, wherein the calibration plate is set at the center of the area surrounded by the multiple cameras.

[0030] In one embodiment, a calibration plate with a white background and a blue circular pattern is used to accurately calculate the scale in the image by measuring the diameter or circumference of the circle, and to correct the distortion in the image to improve the geometric accuracy of the image. Compared with calibration plates of other shapes, circular calibration plates are more tolerant to changes in lighting conditions and viewing angles, and can provide more stable and reliable calibration results.

[0031] In step S204 of this embodiment, the point cloud data corresponding to each camera is determined, including: according to the calibrated internal and external parameters of each camera and the collected image of the object under test, the point cloud data corresponding to each camera is determined by a multi-step phase method and a parallax calculation method.

[0032] In one embodiment, point cloud data corresponding to each camera is determined by a multi-step phase method and a parallax calculation method, including: for any camera among the multiple cameras, a plurality of images of the object under test captured by the camera are obtained, wherein the phases of the grating fringe images projected on the object under test in the multiple images of the object under test are different, and wherein the grating fringe images are sinusoidal grating fringe images; according to the grating fringe images with phase difference in the multiple images of the object under test, a relationship model between the phase and the three-dimensional space coordinates is obtained; according to the two-dimensional coordinates of each pixel point in the multiple images of the object under test, the relationship model between the phase and the three-dimensional space coordinates, and the internal and external parameters of the camera, the point cloud data corresponding to the camera direction is generated.

[0033] By calibrating the intrinsic and extrinsic parameters of the camera, the mapping relationship between the camera coordinate system and the world coordinate system is established, and the pixel coordinates of the camera image are obtained for the subsequent generation of point cloud data.

[0034] Step S206: align the point cloud data corresponding to each camera into the same coordinate system to obtain the point cloud data of the target coordinate system, and perform three-dimensional measurement based on the point cloud data of the target coordinate system.

[0035] Before step S206 of this embodiment, the method further includes: performing at least one of the following data processing on the point cloud data: cleaning, denoising, filtering, and segmentation.

[0036] In one embodiment, the filtering processing of point cloud data mainly includes through filtering and statistical filtering. The role of through filtering is to retain the part of the collected point cloud data required for measurement, and filter out the rest through the value range of the three-dimensional space coordinates, while statistical filtering filters out the noise data between point clouds.

[0037] In step S206 of this embodiment, it includes: iteratively finding the minimum distance between the point cloud data of any two cameras to match the point cloud data, wherein, for the point cloud data of any two cameras, the point cloud data of one camera is regarded as reference point cloud data, and the other is regarded as point cloud data to be registered; determining a transformation matrix based on the matched point cloud data, and performing coordinate transformation on the point cloud data to be registered according to the transformation matrix, so as to convert the point cloud data to be registered from the original coordinate system to the target coordinate system.

[0038] In one embodiment, through operations such as rotation and translation and based on ICP (Iterative Closest Point), the minimum distance between two sets of point cloud data is iteratively found, and a transformation matrix is ​​obtained. The coordinate transformation of the point cloud data to be registered is performed according to the transformation matrix, and it is converted from the original coordinate system to the target coordinate system, that is, the coordinate system corresponding to the reference point cloud data. At the same time, the distance-based deduplication method regards the points with a short distance in the point cloud data to be registered as the same point, and only retains one of the points to remove redundant points, thereby reducing the amount of data and improving the efficiency of point cloud processing. The point cloud data obtained after registration and the remaining unregistered point cloud data are point cloud registered according to the above-mentioned point cloud registration method until all the point cloud data are aligned to the same coordinate system.

[0039] Through the above steps, due to the combination of multiple cameras and structured light, the effects of shadows, occlusions, missing, etc. on the quality of image data can be avoided, and high-quality original image data can be obtained; based on the high-quality image data under multiple viewing angles, the point cloud data corresponding to each camera is determined, and the point cloud data corresponding to each camera is aligned to the same coordinate system to perform fusion of three-dimensional point cloud data, and a three-dimensional measurement result that meets the sub-micron level requirements can be obtained. Therefore, the problem of the need for a three-dimensional measurement method that can meet the sub-micron level accuracy requirements in the related art can be solved, and the effect of improving the three-dimensional measurement accuracy can be achieved.

[0040] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0041] In this embodiment, a three-dimensional measuring device is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0042] Figure 3 is a structural block diagram of a three-dimensional measurement device according to an embodiment of the present application, such as Figure 3As shown, the device includes: a collection module 31, a determination module 32 and a measurement module 33.

[0043] The acquisition module 31 is used to acquire images of the object under test at different viewing angles through multiple cameras, wherein a grating fringe image is projected onto the object under test;

[0044] In one embodiment, the plurality of cameras includes more than two cameras.

[0045] A determination module 32, configured to determine the point cloud data corresponding to each camera according to the calibrated internal and external parameters of each camera and the collected image of the measured object;

[0046] The measurement module 33 is used to align the point cloud data corresponding to each camera into the same coordinate system, obtain the point cloud data of the target coordinate system, and perform three-dimensional measurement based on the point cloud data of the target coordinate system.

[0047] It should be noted that the above modules can be implemented by software or hardware. For the latter, it can be implemented in the following ways, but not limited to: the above modules are all located in the same processor; or the above modules are located in different processors in any combination.

[0048] To facilitate the understanding of the technical solution provided by the present application, a detailed explanation will be given below in combination with embodiments of specific scenarios.

[0049] Common 3D machine vision measurement methods include laser measurement, monocular structured light measurement, and binocular measurement. The following is a description of three common 3D machine vision measurement methods and the shortcomings of each method;

[0050] Laser measurement can obtain measurement results with micron-level accuracy and is not affected by the color and texture of the object being measured. It is suitable for a wide range of measurement tasks and can obtain a large amount of data in a short time. However, laser measurement is sensitive to ambient light and needs to be measured in a relatively stable environment. The equipment cost is high and there are high site requirements.

[0051] Monocular structured light measurement can achieve fast, non-contact measurement with an accuracy range of millimeters to sub-millimeter levels. It is suitable for three-dimensional measurement of small and medium-sized objects, and the structured light measurement equipment used is relatively cheap and easy to promote and apply. However, the monocular structured light measurement method is easily affected by the surface material and reflectivity of the object being measured, and parameters need to be adjusted for different objects.

[0052] The binocular measurement method can obtain image data at different viewing angles to obtain relatively comprehensive information, and is suitable for three-dimensional measurement of various objects. The accuracy range is at the millimeter level, but precise calibration and registration are required, and the accuracy is affected by factors such as ambient light and noise, and it requires a relatively long measurement time.

[0053] In addition to their own shortcomings, the above three measurement methods are unable to simultaneously meet the requirements of sub-micron accuracy and low-cost competitiveness.

[0054] If we want to meet both sub-micron precision requirements and low-cost competitiveness requirements, we can consider it from the perspective of data acquisition design, use common standard components to build a high-resolution, multi-perspective acquisition terminal, and obtain high-quality, dense, and complete point cloud data for application function development, meeting sub-micron and low-cost requirements.

[0055] The embodiment of the present application proposes a 3D high-precision measurement method based on 4-eye structured light, which uses 4-eye high-speed and high-resolution CCD image sensors and lenses to image the reflected light of the projected area array stripe structured light from multiple directions. It can solve the common problems of unilateral shadows and point cloud missing in the work of traditional monocular structured light 3D cameras / line laser 3D cameras, output a more dense and complete 3D point cloud image, and then achieve high-precision measurement of related positions by preprocessing the point cloud data, plane fitting, distance calculation, etc., thereby meeting the technical requirements of high-precision 3D visual detection at the sub-micron level and low cost.

[0056] The embodiments of the present application can be applied to the quality inspection of components in the 3C electronics industry. In addition, it can also be applied to the manufacture and testing of aerospace parts and equipment, the manufacture and quality control of automobile parts, various experiments and measurements in scientific research, etc.

[0057] The 3D high-precision measurement method based on 4-eye structured light can be applied to high-precision measurement fields such as stereoscopic dimension measurement, spatial precise positioning guidance, and device surface shape analysis in the 3C electronic manufacturing industry. It is suitable for workpiece measurement within a depth of field of 70mm and a field of view of 75x30mm. In the embodiment of this application, there are certain requirements for hardware computing power. The stronger the computing power, the shorter the running time, which can meet the scene requirements of high work beats.

[0058] Figure 4 is a structural block diagram of a 3D high-precision measurement system according to an embodiment of the present application, such as Figure 4 As shown, the system mainly includes five modules: hardware construction, parameter calibration, point cloud construction, point cloud processing, and point cloud registration.

[0059] Hardware construction is responsible for the selection and assembly of cameras, structured light projectors, industrial computers, data cables, acquisition cards, calibration boards and other equipment, as well as calibration and debugging. It is the basis for 3D visual measurement. Based on the constructed hardware system, high-quality image data can be obtained, thereby achieving accurate three-dimensional measurement. Correct selection and optimization can improve the accuracy and efficiency of measurement, and can also provide customized solutions for different application scenarios.

[0060] Parameter calibration is used to determine the internal and external parameters of devices such as cameras and laser scanners in order to perform accurate 3D measurements. It usually includes two aspects: camera internal parameter calibration and external parameter calibration. Camera internal parameter calibration mainly refers to determining the camera's focal length, distortion and other parameters, which will affect the accuracy and quality of camera imaging. External parameter calibration refers to determining the camera's position and posture and other parameters. Correct parameter calibration can eliminate the error of the device itself, thereby achieving more accurate 3D measurement.

[0061] Point cloud construction is used to convert discrete point data of a three-dimensional scene or object surface into a point cloud model, scan and capture object surface image information through industrial cameras and structured light, and generate point cloud data using the phase shift method.

[0062] Point cloud processing is the process of processing and analyzing the collected point cloud data, extracting useful information and converting it into a visualization or digital model. Point cloud processing includes: data representation, statistical filtering, Euclidean clustering segmentation, MLS, point cloud downsampling and other functions.

[0063] Point cloud registration is the process of aligning multiple point cloud data into the same coordinate system for subsequent processing and analysis, including reference point cloud selection, feature matching, calculation of transformation matrix, accuracy assessment and other functions.

[0064] Figure 5 is a flow chart of a 3D high-precision measurement method based on 4-eye structured light according to an embodiment of the present application, such as Figure 5 As shown, the method comprises the following steps:

[0065] S501, build the hardware environment, that is, use the DMD module to project coded structured light and four high-resolution industrial cameras to collect image data;

[0066] For example: deploy four high-resolution industrial cameras in the middle of the four sides in a square form, and deploy a DMD structured light projector in the middle of the square to image the workpieces within the public field of view;

[0067] S502, calibrating the camera internal parameters and system parameters with the aid of a calibration plate, and then projecting the grating fringe pattern onto the surface of the object to be measured to obtain the relationship between the phase and the three-dimensional space coordinates.

[0068] Specifically, a circular-patterned calibration plate is used to calibrate the internal parameters and system parameters of four high-resolution industrial cameras, and a mapping relationship between the camera coordinate system and the world coordinate system is established to obtain the camera image pixel coordinates. Sinusoidal stripes are generated by computer programming, and the sinusoidal stripes are projected onto the object to be measured through a projection device. A CCD (i.e., industrial camera) camera is used to capture the degree of bending of the stripes modulated by the object, and the bent stripes are demodulated to obtain the phase, which is then converted into spatial height information, so as to obtain a relationship model between the desired question and the three-dimensional spatial coordinates.

[0069] In one embodiment, a calibration plate with a white background and a blue circular pattern is designed based on the reflection of the object surface and the recognition and positioning accuracy of the pattern. By measuring the diameter or circumference of the circle, the scale in the image can be accurately calculated, and the distortion in the image, such as radial and tangential distortion, can be corrected to improve the geometric accuracy of the image. By shooting the calibration plate at different viewing angles (i.e., four camera viewing angles) and analyzing the pixel coordinates of the marking points, the position and posture of the camera can be calculated, and then three-dimensional reconstruction and depth estimation can be performed.

[0070] Compared with calibration plates of other shapes, circular calibration plates are more tolerant to changes in lighting conditions and viewing angles, and can provide more stable and reliable calibration results.

[0071] The projection module generates a sinusoidal grating fringe pattern with a certain phase shift through computer programming, and projects the sinusoidal grating fringe pattern onto the object to be measured, and then the camera synchronously captures a set of grating images. By analyzing this set of grating images with a certain phase difference, the initial phase value of the surface morphology data of the object to be measured is solved. Phase unfolding is performed through the correct initial phase value and more coding patterns to obtain an unambiguous absolute phase value within the image range. Usually the light intensity with phase shift can be expressed as:

[0072] In(x, y) = A(x, y) + B(x, y)cos [Φ(x, y) + an] Formula (1)

[0073] In this embodiment, a four-step phase shift method is adopted, and the phase shift steps are: 0, π / 2, π, 3π / 2, then the light intensity expressions are:

[0074] In(x, y) = A(x, y) + B(x, y)cos [Φ(x, y)]; Formula (2)

[0075] In(x, y) = A(x, y) + B(x, y)cos[Φ(x, y) +π / 2]; Formula (3)

[0076] In(x, y) = A(x, y) + B(x, y)cos [Φ(x, y) + π]; Formula (4)

[0077] In(x, y) = A(x, y) + B(x, y)cos[Φ(x, y) +3π / 2]; Formula (5)

[0078] By performing phase extraction on four grating images with different phase shifts, the following formula (6) can be used to calculate the main phase value.

[0079] Φ(x, y) = arctan( (I2 - I4) / (I1 - I3) ) Formula (6)

[0080] Four sinusoidal light intensities with different phase shifts (i.e. Figure 6 (1) to (4)) in the above table are displayed in the same coordinate system. Figure 6 As shown in FIG. 1 , by solving the image phase principal value based on the above four-step phase shift method, the phase principal value diagram can be obtained, as shown in FIG. Figure 7 As shown in the figure, the phase principal value changes periodically in the image. However, since the phase principal value obtained by a single frequency cannot completely cover the pixel distribution of the entire image, pixels at different positions may have the same phase principal value. Therefore, the phase principal value needs to be expanded to obtain the absolute phase that can cover the entire image.

[0081] S503, point cloud data generation, obtaining point cloud data under each camera perspective through multi-step phase method and parallax calculation;

[0082] Through the relationship model between the camera image pixel coordinates and phase and the three-dimensional space coordinates obtained in the calibration process, using the known pixel coordinates (m, n, θ) and a1-a8 (i.e., the intrinsic and extrinsic parameters of the camera), through the combined model formula and the camera internal and external parameter formula, we can get the relationship between Xc, Yc, and Zc respectively about θ (the brightness value of the pixel point) and the intrinsic parameter, thereby calculating the three-dimensional space coordinates of the corresponding pixel and generating point cloud data.

[0083] S504, point cloud preprocessing, performing at least one of the following operations on the original point cloud data (i.e., the point cloud data obtained in step S503): cleaning, denoising, filtering, downsampling, and segmentation.

[0084] Point cloud data processing mainly includes through filtering and statistical filtering. The role of through filtering is mainly to retain the part of the point cloud required for measurement, and filter out the rest through the value range of the three-dimensional space coordinates (X, Y, Z). Statistical filtering filters out outliers (i.e., noise data) between point clouds, downsamples the point cloud data, and adjusts the point cloud density.

[0085] In one embodiment, a statistical filtering algorithm is used to perform a statistical analysis on the neighborhood of each point, and based on the distance distribution characteristics from the target point to all its neighboring points, some outliers that do not meet the requirements are filtered out. The algorithm iterates the entire input twice: in the first iteration, the average distance from each point to its nearest k neighboring points is calculated, and the result obtained conforms to the Gaussian distribution; then the average value u and standard deviation σ of all average distances are calculated to determine the distance threshold thresh_d, and thresh_d=u±kσ, k is the standard deviation multiplier; in the second iteration, if the average neighborhood distance of these points is lower or higher than the above threshold, these points will be classified as inliers or outliers.

[0086] S505, point cloud registration, completes the precise registration of the four sets of point cloud data through operations such as rotation, translation, and deduplication.

[0087] In one embodiment, for any two sets of point cloud data, one set of point cloud data is used as reference point cloud data, and the other set of point cloud data is used as point cloud data to be registered. Based on ICP (Iterative Closest Point, iterative closest point algorithm), the minimum distance between the two sets of point cloud data is iteratively found through operations such as rotation and translation, and a transformation matrix is ​​obtained. According to the transformation matrix, the coordinate transformation of the point cloud data to be registered is performed, and it is converted from the original coordinate system to the target coordinate system (i.e., the coordinate system corresponding to the reference point cloud data). At the same time, the distance-based deduplication method regards the points with a relatively close distance in the point cloud data to be registered as the same point, and only retains one of the points to remove redundant points, thereby reducing the amount of data and improving the efficiency of point cloud processing.

[0088] The point cloud data obtained after registration and the remaining unregistered point cloud data are registered according to the above point cloud registration method until all the point cloud data are aligned to the same coordinate system.

[0089] S506, measurement application, extracting key features to realize application functions such as height distance calculation and coplanarity measurement.

[0090] In one embodiment, the parameters of a data model containing noise and outliers are estimated through an iterative algorithm based on random sampling of RANSAC, so as to fit the corresponding plane equation, perform spacing calculation on the obtained plane equation, and obtain the distance difference between each plane, which is applied to various measurement calculations.

[0091] Among them, for the plane calculation process, it is first necessary to select the point cloud data through ROI (Region of Interest) to obtain the point cloud data to be fitted; then use the plane equation ax+by+cz+d=0 in the space as the model equation, and use the fitting algorithm to fit it to obtain the corresponding plane equation.

[0092] The principle of RANSAC algorithm is as follows:

[0093] Step S1, selecting the minimum data sample of the estimation model, for example: for two-dimensional or three-dimensional straight line fitting, at least 2 points are required to determine a straight line; for three-dimensional plane fitting, at least 3 points are required to determine a three-dimensional plane.

[0094] Step S2, using the minimum data sample, calculates the fitting model, that is, the equation of the line or the equation of the plane.

[0095] Step S3, bringing all models into the fitted model, and calculating the number of inliers, that is, the number of data points whose errors with the fitted model are within a certain threshold range.

[0096] Step S4, comparing the number of inliers of the current model and the best model obtained in the previous iteration, recording the model parameters and the number of inliers of the model with the largest number of inliers, wherein the more inliers there are, the better the model is.

[0097] Repeat steps S1-S4 until the iteration termination condition is reached, such as the maximum number of iterations is reached or the number of internal points reaches the iteration termination condition.

[0098] Finally, after obtaining the plane equations of each plane, the height of each plane can be calculated. In the process of calculating the coplanarity, considering the impact of data fluctuations and noise on the results, in this embodiment, 10% of the highest points and 10% of the lowest points are filtered out, and then the shortest distance is subtracted from the farthest distance.

[0099] Through the above-mentioned embodiments of the present application, four industrial cameras combined with structured light are used to obtain high-quality original image data, avoiding shadows, occlusions, missing phenomena and the like; multi-view 3D point cloud fusion technology based on multi-eye stereo imaging technology is used to further improve the measurement repeatability accuracy and meet sub-micron requirements; common basic standard components are used, and through solution design, the expected effect is achieved, costs are reduced, and product competitiveness is improved.

[0100] The technical solutions in the embodiments of the present application can be replicated in other scenarios such as 3C electronics, semiconductors, and precision machining, and can effectively improve the accuracy and reliability in the detection process. At the same time, the non-contact measurement method can also improve the measurement efficiency and the possibility of automated production.

[0101] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above method embodiments when running.

[0102] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0103] An embodiment of the present application further provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0104] In an exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0105] For specific examples in this embodiment, reference may be made to the examples described in the above embodiments and exemplary implementation modes, and this embodiment will not be described in detail herein.

[0106] Obviously, those skilled in the art should understand that the above modules or steps of the present application can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order from that herein, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present application is not limited to any specific combination of hardware and software.

[0107] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the principles of the present application shall be included in the protection scope of the present application.

Claims

1. A three-dimensional measurement method, characterized in that: include: The images of the object to be measured are collected at different viewing angles by using multiple cameras, wherein a grating fringe image is projected onto the object to be measured; Determine the point cloud data corresponding to each camera according to the calibrated internal and external parameters of each camera and the collected image of the object to be measured; The point cloud data corresponding to each camera is aligned to the same coordinate system to obtain the point cloud data of the target coordinate system, and three-dimensional measurement is performed based on the point cloud data of the target coordinate system.

2. The method according to claim 1, characterized in that in, The multiple cameras are four cameras, and the four cameras have the same spacing between them and the same distance from the measured object.

3. The method according to claim 1, characterized in that According to the calibrated internal and external parameters of each camera and the collected image of the measured object, the method further includes: The internal parameters and external parameters of each camera are calibrated by a calibration plate, so that each camera corrects the distortion in the captured image based on the calibrated internal parameters and external parameters, wherein the calibration plate is arranged at the center of the area surrounded by the multiple cameras.

4. The method according to claim 1, characterized in that Determine the point cloud data corresponding to each camera, including: According to the calibrated internal and external parameters of each camera and the collected image of the object to be measured, the point cloud data corresponding to each camera is determined by a multi-step phase method and a parallax calculation method.

5. The method according to claim 4, characterized in that The point cloud data corresponding to each camera is determined by the multi-step phase method and parallax calculation method, including: For any camera among the multiple cameras, a plurality of images of the object under test captured by the camera are acquired, wherein the phases of the grating fringe images projected on the object under test in the multiple images of the object under test are different, and the grating fringe images are sinusoidal grating fringe images; Acquire a relationship model between phase and three-dimensional space coordinates according to the grating fringe images with phase difference in the plurality of images of the measured object; Point cloud data corresponding to the camera direction is generated according to the two-dimensional coordinates of each pixel point in the multiple images of the measured object, the relationship model between the phase and the three-dimensional space coordinates, and the internal and external parameters of the camera.

6. The method according to claim 1, characterized in that Before aligning the point cloud data corresponding to each camera into the same coordinate system, the method further includes: The point cloud data is subjected to at least one of the following data processing steps: cleaning, denoising, filtering, and segmentation.

7. The method according to claim 1, characterized in that Aligning the point cloud data corresponding to each camera into the same coordinate system includes: Matching the point cloud data is performed by iteratively finding the minimum distance between the point cloud data of any two cameras, wherein, for the point cloud data of any two cameras, the point cloud data of one camera is regarded as the reference point cloud data, and the other is regarded as the point cloud data to be registered; A transformation matrix is ​​determined based on the matched point cloud data, and coordinate transformation is performed on the point cloud data to be registered according to the transformation matrix, so as to convert the point cloud data to be registered from an original coordinate system to a target coordinate system.

8. A three-dimensional measuring device, characterized in that: include: An acquisition module, used to acquire images of the object under test at different viewing angles through multiple cameras, wherein a sinusoidal-fringe structured light is projected onto the object under test; A determination module, used to determine the point cloud data corresponding to each camera according to the calibrated internal and external parameters of each camera and the collected image of the measured object; The measurement module is used to align the point cloud data corresponding to each camera into the same coordinate system to obtain the point cloud data of the target coordinate system, and perform three-dimensional measurement based on the point cloud data of the target coordinate system.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program implements the steps of the method described in any one of claims 1 to 7 when executed by a processor.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method described in any one of claims 1 to 7 are implemented.

Images