Three-dimensional shape measurement method based on space-time interlaced acquisition structured light

Abstract

The invention relates to a three-dimensional shape measurement method based on space-time staggered acquisition of structured light, which comprises the following steps: placing a measured object on a translation table capable of horizontally moving, and projecting a structured light image to the surface of the measured object through a digital projection system to form an envelope structured light image; collecting image information of a corresponding position of the surface of the measured object through a linear array camera; in the process that the measured object moves along with the translation stage, the digital projection system is controlled to switch projection images every time the measured object moves by a first distance, and the line-scan digital camera is synchronously triggered to collect image information; and combining the images acquired by the linear array cameras to form a plurality of camera shooting structured light images, resolving the camera shooting structured light images, and decoding the position information of each point on the surface of the measured object in combination with a ray tracing technology to obtain a three-dimensional shape measurement result of the measured object. Compared with the prior art, the high-precision three-dimensional shape reconstruction of the high-speed moving object can be realized, and the method has the advantages of high measurement precision, high robustness and the like.

Description

Technical Field

[0001] The present invention relates to the field of three-dimensional vision technology, and in particular to a three-dimensional shape measurement method based on time-space staggered acquisition of structured light. Background Art

[0002] As an important achievement of the integration of automation and intelligence, machine vision technology has become a new standard in the field of industrial inspection. As a part of traditional machine vision, two-dimensional vision has been widely used in many industrial inspection and automation applications, but the lack of depth information seriously affects the accuracy and completeness of its inspection results. In contrast, three-dimensional vision technology can achieve more comprehensive and accurate inspection and analysis by obtaining the three-dimensional shape and position information of objects, and is now playing an increasingly important role in the field of machine vision.

[0003] From a technical perspective, whether 3D vision technology can meet the growing demand for industrial automation depends on how to achieve fast and high-precision 3D shape measurement of objects with different optical properties. It must meet the rhythm requirements of industrial production and the standards of high-precision measurement. High precision ensures product quality, while high-speed processing capabilities ensure production efficiency. Both are indispensable, otherwise they will have an adverse impact on the effect and efficiency of industrial production. Among the many 3D shape measurement technologies, stripe structured light technology accurately encodes the position information of each point on the surface of the object by projecting stripe information with strong texture features on the surface of the object, and then the camera collects these projected images. By performing phase resolution on the stripe image and combining it with triangular ray tracing technology, the position information of each point on the surface of the object is decoded to obtain the 3D shape information of the object. This technology has the advantages of high precision, high resolution, non-contact measurement and strong sample adaptability, and is particularly suitable for 3D shape measurement in industrial automation.

[0004] However, when faced with high-speed, dynamically changing assembly line scenes, this technology is easily affected by motion blur and fringe distortion, making it difficult to achieve high-precision 3D measurement. Therefore, although fringe structured light technology has significant advantages in measurement accuracy and surface adaptability, it is greatly limited in application scenarios such as high-speed assembly lines because the object needs to stop during the measurement process, making it difficult to further promote.

[0005] The fringe structured light 3D topography measurement technology cannot meet the needs of high-speed moving objects. The main reason is that the technology relies on a large-target area array camera to collect multiple fringe images with different phases to obtain the complete 3D topography of the object. The entire measurement process usually takes tens to hundreds of milliseconds to acquire images. For high-speed moving objects, their topography has shifted significantly over such a long time span. Therefore, this displacement may cause serious ghosting in the measurement results, or even cause the phase solution to fail, which greatly reduces the measurement accuracy and reliability.

[0006] Therefore, there is an urgent need for a technical solution to shorten the image acquisition time, solve the problem of object misalignment during the image acquisition process, and ultimately realize the stripe structured light measurement of high-speed moving objects. Summary of the invention

[0007] The purpose of the present invention is to provide a three-dimensional morphology measurement method based on time-space interlaced acquisition of structured light in order to overcome the defects of the above-mentioned prior art that the fringe image is collected by an area array camera, the entire measurement process usually requires tens to hundreds of milliseconds of image acquisition time, and the morphology has undergone significant displacement, thereby reducing the measurement accuracy and reliability.

[0008] The purpose of the present invention can be achieved by the following technical solutions:

[0009] A three-dimensional shape measurement method based on time-space staggered acquisition of structured light, comprising the following steps:

[0010] The object to be measured is placed on a horizontally movable translation stage, and a structured light image is projected onto the surface of the object to be measured through a digital projection system to form an envelope structured light image; the image information on the corresponding envelope line of the surface of the object to be measured is collected through a line array camera;

[0011] The digital projection system needs to project structured light images containing N images, namely S1, S2, …, S N When the object to be measured moves along with the translation stage, each time it moves a first distance, the digital projection system is controlled to cyclically switch the N structured light images S1, S2, ..., S N , and synchronously trigger the linear array camera to collect image information;

[0012] The images collected by the line array camera are recombined to form projected structured light images S1, S2, ..., S N The corresponding N cameras take structured light images I1, I2, ..., I N , the camera takes structured light images I1, I2, ..., I N The solution is performed and combined with ray tracing technology, the position information of each point on the surface of the measured object is decoded to obtain the three-dimensional shape measurement result of the measured object.

[0013] Furthermore, the projected structured light image is a cosine fringe structured light image, and the N cameras take structured light images I1, I2, ..., I N The specific process of solving is as follows:

[0014] According to the N cameras taking structured light images I1, I2, ..., I N Performing a simultaneous solution to obtain a wrapped phase, and using an unwrapping algorithm to unfold the wrapped phase into a continuous phase distribution, so as to correspond the pixels of the linear array camera with the pixels of the digital projection system, and further determine the three-dimensional coordinates of each pixel corresponding to the surface of the object under test;

[0015] Sequentially perform corresponding processing of the three-dimensional coordinates of the surface of the object to be measured on different pixels of the image information collected by the linear array camera, and obtain the three-dimensional coordinate information of each point on the surface of the object along the envelope line corresponding to the image information;

[0016] According to the image information on different envelope lines, the corresponding three-dimensional coordinate information of each point is obtained respectively, and combined to reconstruct the complete three-dimensional shape information of the measured object.

[0017] Furthermore, the unwrapping algorithm is a phase solution method based on a deep convolutional neural network, and the specific process includes:

[0018] A deep convolutional neural network is used to capture the structured light images I1, I2, ..., I N The package count corresponding to each pixel in is classified and predicted, and then the package count is multiplied by 2π and added to the package phase to obtain the continuous phase distribution.

[0019] Furthermore, the projected structured light image is a Gray code structured light image, and the N cameras take structured light images I1, I2, ..., I N The calculation process is to calculate the captured Gray code image, and then combine the ray tracing technology to decode the position information of each point on the surface of the object to be measured, and obtain the three-dimensional shape measurement result of the object to be measured.

[0020] Furthermore, the ray tracing technology is specifically:

[0021] According to the N cameras taking structured light images I1, I2, ..., I N The acquisition time corresponding to each line array image in the image is obtained, and the ray tracing path collected by each pixel of the line array camera under different acquisition times is obtained to assist in solving the position information of each point on the surface of the measured object.

[0022] Furthermore, the digital projection system comprises:

[0023] Digital micromirror array, used to control the state of light and generate binary images;

[0024] A beam projection system, used for adjusting the binary image into a cosine fringe image;

[0025] The light beam projection system comprises a light source, a collimator and a projection lens group. The light beam emitted by the light source is collimated into parallel light by the collimator and irradiated onto the surface of the digital micromirror array. After being reflected by the digital micromirror array, the light beam enters the projection lens group to form a cosine fringe image.

[0026] The projection lens group comprises a tube lens, a filter aperture and an objective lens which are sequentially distributed. The filter aperture is located at the focal plane of the tube lens and the objective lens and is used for frequency domain filtering of the light field modulated by the digital micromirror array.

[0027] Furthermore, the digital projection system comprises:

[0028] Liquid crystal spatial light controller, used to control the state of light and generate structured light images;

[0029] A beam projection system, used to project a structured light image onto the surface of the object being measured;

[0030] The light beam projection system includes a light source, a collimator and a projection lens group. The light beam emitted by the light source is collimated into parallel light by the collimator and irradiated onto the surface of the liquid crystal spatial light manipulator. After being reflected by the liquid crystal spatial light manipulator, the light beam enters the projection lens group to form a structured light image.

[0031] Furthermore, before the method performs three-dimensional shape measurement on the object to be measured, the method also includes calibrating parameters of a three-dimensional shape measurement system composed of a translation stage, a digital projection system and a linear array camera;

[0032] The parameter calibration process is specifically as follows:

[0033] A checkerboard calibration plate is placed on the translation stage, the checkerboard calibration plate is tilted at different angles, and after the checkerboard calibration plate is driven to move by the translation stage, a line array camera is used to photograph row by row to obtain a set of calibration images of the checkerboard calibration plate; and internal and external parameter values ​​of the line array camera and its imaging lens group are obtained through the calibration image and the geometric optical transformation relationship;

[0034] A set of binary images for calibration is projected through a digital projection system, the checkerboard calibration plate is tilted at different angles, and a series of different projection images are obtained by taking pictures with a line array camera, so as to calculate the internal and external parameter values ​​of the digital projection system.

[0035] Furthermore, the parameter calibration process also includes:

[0036] Standard samples with different surface optical properties are manufactured, and the three-dimensional shape measurement system is used to perform three-dimensional shape measurement on the standard samples to obtain measurement results, which are compared with the corresponding design values ​​of the standard samples, thereby verifying the measurement accuracy and robustness of the three-dimensional shape measurement system.

[0037] The present invention also provides a three-dimensional shape measurement device based on time-space staggered acquisition of structured light, comprising:

[0038] A translation stage, used to carry the object to be measured;

[0039] A digital projection system is used to project the structured light image onto the surface of the object to be measured to form an envelope structured light image;

[0040] Linear array camera, used to collect image information on the envelope line corresponding to the surface of the object being measured;

[0041] The controller is connected to the digital projection system and the linear array camera respectively, and is used to execute the above-mentioned three-dimensional morphology measurement method based on the spatiotemporal staggered acquisition of structured light.

[0042] Compared with the prior art, the present invention has the following advantages:

[0043] (1) The present invention proposes a spatiotemporal staggered acquisition method based on a line array camera and high-speed digital projection technology. By replacing the area array camera with a line array camera and combining it with high-speed digital projection technology, the microsecond switching of the fringe structured light projection image can be achieved. The time for acquiring fringe patterns with different phases can be shortened from tens of milliseconds to tens of microseconds, thereby solving the problem of object misalignment during image acquisition, ghosting and fringe distortion during measurement of high-speed moving objects, and realizing high-precision three-dimensional measurement in a high-speed dynamic environment.

[0044] (2) The present invention optimizes the traditional phase solution algorithm through a lightweight deep learning network, which not only simplifies the traditional complex phase solution steps, but also uses the characteristics of deep neural networks to automatically learn and extract high-level features from the original image, greatly improving the system's noise resistance in dynamic environments and ensuring that the three-dimensional morphology can still be accurately solved in low signal-to-noise ratio images.

[0045] (3) The present invention designs and optimizes the projection technology of cosine fringe images, adopts a digital micromirror array with a fast control speed to control the state of light, and combines the light beam projection system to control the binary image projected by the digital micromirror array into a cosine image, thereby improving the image switching speed of the system and achieving a high frame rate of 60,000 frames per second, ensuring that measurement data can still be accurately captured under high-speed motion conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1A schematic flow chart of a three-dimensional shape measurement method based on spatiotemporal staggered acquisition of structured light provided in an embodiment of the present invention;

[0047] Figure 2 A schematic diagram of a data processing process of a three-dimensional shape measurement method based on spatiotemporal staggered acquisition of structured light provided in an embodiment of the present invention;

[0048] Figure 3 A schematic diagram of a cosine fringe beam projection structure based on a digital micromirror array provided in an embodiment of the present invention;

[0049] Figure 4 Schematic diagram of the process of the phase solution method provided in an embodiment of the present invention, wherein (a) is a schematic diagram of the wrapped phase, (b) is a schematic diagram of the real phase, and (c) is a schematic diagram of the phase solution principle based on deep learning;

[0050] Figure 5 Schematic diagram of the structure of a stripe structured light high-speed three-dimensional shape measurement system provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0051] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0052] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0053] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0054] Example 1

[0055] like Figure 1 As shown, this embodiment provides a three-dimensional shape measurement method based on time-space staggered acquisition of structured light, including the following steps:

[0056] S1: The object to be measured is placed on a horizontally movable translation stage, and a structured light image is projected onto the surface of the object to be measured through a digital projection system to form an envelope structured light image; the image information on the corresponding envelope line of the surface of the object to be measured is collected through a line array camera;

[0057] S2: The digital projection system needs to project structured light images containing N images, namely S1, S2, …, S N When the object to be measured moves with the translation stage, each time it moves a first distance, the digital projection system is controlled to cyclically switch the projected N structured light images S1, S2, ..., S N , and synchronously trigger the linear array camera to collect image information;

[0058] S3: Recombining the images collected by the linear array camera to form projected structured light images S1, S2, ..., S N The corresponding N cameras take structured light images I1, I2, ..., I N , the camera takes structured light images I1, I2, ..., I N The solution is performed and combined with ray tracing technology, the position information of each point on the surface of the measured object is decoded to obtain the three-dimensional shape measurement result of the measured object.

[0059] Optionally, the projected structured light image is a cosine fringe structured light image, and the N cameras take structured light images I1, I2, …, I N The specific process of solving is as follows:

[0060] According to the N cameras taking structured light images I1, I2, ..., I N Performing a simultaneous solution to obtain a wrapped phase, and using an unwrapping algorithm to unfold the wrapped phase into a continuous phase distribution, so as to correspond the pixels of the linear array camera with the pixels of the digital projection system, and further determine the three-dimensional coordinates of each pixel corresponding to the surface of the object under test;

[0061] Sequentially perform corresponding processing of the three-dimensional coordinates of the surface of the object to be measured on different pixels of the image information collected by the linear array camera, and obtain the three-dimensional coordinate information of each point on the surface of the object along the envelope line corresponding to the image information;

[0062] According to the image information on different envelope lines, the corresponding three-dimensional coordinate information of each point is obtained respectively, and combined to reconstruct the complete three-dimensional shape information of the measured object.

[0063] Optionally, the projected structured light image is a Gray code structured light image, and the N cameras take structured light images I1, I2, …, I NThe calculation process is to calculate the captured Gray code image, and then combine the ray tracing technology to decode the position information of each point on the surface of the object to be measured, and obtain the three-dimensional shape measurement result of the object to be measured.

[0064] Equivalent to, Figure 2 As shown, in this method, the object to be measured is placed on a high-speed translation stage, and the digital projection system projects the cosine fringe image onto the surface of the object through a projection lens group to form a fringe envelope with strong feature information. The linear array camera collects image information on the corresponding envelope line of the object surface through an imaging lens. When the object moves at high speed along the translation stage, the encoder will simultaneously send a synchronization signal to the linear array camera and the digital projection system for each movement of a distance Δx, control the projection system to switch the projected image, and synchronously trigger the linear array camera to collect the corresponding image. Therefore, when the object moves a certain distance, the images collected by the linear array camera can be combined to form several fringe images, and these images have different initial phases. By solving these fringe images and combining them with ray tracing technology, the three-dimensional morphology measurement of the object can be achieved.

[0065] Preferably, the number of phase shifting steps can be selected as three steps, which minimizes the number of phase shifting steps while ensuring the measurement accuracy. Therefore, the sinusoidal fringes projected by the digital projection system are three sinusoidal fringe images with an initial phase difference of 2π / 3, and the three images are switched in sequence. At this time, three fringe images I1(x, y), I2(x, y) and I3(x, y) with fixed phase differences can be collected by scanning the line array camera line by line, that is,

[0066]

[0067] Where I(x,y) is the background intensity, A(x,y) is the modulation intensity of the fringe pattern, and φ(x,y) is the phase distribution to be calculated. After calculating formula X, the phase φ(x,y) can be obtained as:

[0068]

[0069] Since the output range of the arctan function is [-π,π], the calculated phase is a wrapped phase, and the phase value will repeat within this range. If a continuous phase distribution is required, an unwrapping algorithm is needed to unfold the wrapped phase into a continuous phase value distribution.

[0070] After obtaining the continuous phase distribution, the pixels of the line array camera can be directly matched with the pixels in the digital projection system. In order to obtain the three-dimensional coordinates of the surface of the object to be measured, the coordinate system is first transformed. Assuming that the object remains stationary, it is equivalent to the translation of the line array camera and the digital projection system during the imaging process. When the system is in a certain position, each pixel of the line array camera is analyzed. First, a straight line is determined by connecting the pixel with the center of the imaging lens, and the stripe line projected in the digital projection system by the phase value corresponding to the pixel is found. Using the three-dimensional geometric relationship to obtain the intersection of the two, the three-dimensional coordinates of the object surface corresponding to the pixel can be obtained.

[0071] By sequentially scanning different pixels of the line array camera, the three-dimensional coordinate information of each point on the surface of the object along the envelope line at that position can be obtained. When the line array camera and the digital projection system are at different positions of the object, a series of envelope lines of the surface of the object at different positions can be obtained. By combining these envelope lines, the complete three-dimensional shape information of the measured object can be reconstructed.

[0072] In order to verify the above method, preferably, this embodiment will construct a simulation model to simulate the entire image acquisition process and the three-dimensional information solution process. First, a three-dimensional surface of an object to be measured is generated by Zernike polynomials, and is placed under the projector as the object to be measured in the simulation system. Subsequently, a fixed cosine fringe pattern is projected by a projector through geometric ray tracing, and the modulated structured light image when the camera shoots the object moving to different positions along a certain direction is simulated. As the shape and position of the object surface change, the collected image information will change accordingly. Finally, based on the proposed three-dimensional information solution method, the three-dimensional morphology of the object is calculated. The measurement results obtained by simulation are compared with the true value information set in the simulation, and the error distribution is analyzed to verify the effectiveness of the method.

[0073] The key technologies of the above method include high-fidelity cosine fringe image high-speed generation technology based on digital projection technology, fringe image phase solution algorithm with high speed and high noise resistance, and internal and external parameter calibration technology of high-speed digital projection system and linear array camera. The following is a detailed description of each part:

[0074] 1. High-speed generation technology of high-fidelity cosine fringe images based on digital projection technology

[0075] The core of digital projection technology is to use digital micromirror array (DMD) to control the state of light, which has the advantages of large number of pixels and fast control speed, and can reach a control speed of up to 10,000 Hz. However, due to the limitation of the control principle, the control of light by this technology can only produce binary images, and cannot be directly used in the generation of cosine fringe images required by this method.

[0076] In order to adjust the binary image projected by the digital micromirror array to a cosine image, this embodiment adopts the following method: Figure 3 The light beam projection system shown includes a light source, a collimator and a projection lens group. The light beam emitted by the light source is collimated into parallel light by the collimator and irradiated onto the surface of the digital micromirror array. After being reflected by the digital micromirror array, the light beam enters the projection lens group to form a cosine fringe image.

[0077] The projection lens group includes a tube lens, a filter diaphragm and an objective lens which are distributed in sequence. The filter diaphragm is located at the focal plane position of the tube lens and the objective lens and is used for frequency domain filtering of the light field modulated by the digital micromirror array.

[0078] The system uses a collimator to collimate the light beam emitted by the LED (light source) into parallel light and irradiate it onto the surface of the digital micromirror array. After being reflected by the digital micromirror array, the light beam enters the projection lens group consisting of a tube lens, an objective lens, and a filter aperture. The filter aperture is located at the focal plane of the tube lens and the objective lens, and is used to perform frequency domain filtering on the light field modulated by the digital micromirror array.

[0079] In order to ensure the filtering performance of the system, this embodiment simulates the light field propagation of the entire projection optical system. First, the LED light source is discretized into several different wavelengths, and the light of each wavelength is simulated separately. The simulation adopts the light field angular spectrum propagation method, and the simulation results of each wavelength are integrated and synthesized at the image plane. By adjusting the size of the filter aperture, the fidelity between the projection fringes and the standard cosine fringes is used as the optimization target, and finally the optimal filter aperture size is determined.

[0080] In addition, it should be noted that since the digital micromirror array works at a very high frame rate, the exposure time of the camera is also very short, and the flipping process of the micromirror actually takes a certain amount of time. This means that during the camera exposure, the micromirror array gradually flips from one state to another. In this process, it is necessary to clarify whether the flipping of the micromirror will affect the projected cosine fringe image, and to what extent.

[0081] Preferably, in order to solve this problem, the present embodiment introduces the influence of the micromirror deflection angle on the light field propagation in the light field propagation model, and simulates the change of the cosine fringe image projected at different deflection angles. Then, according to the micromirror flipping process, the image at each deflection angle is integrated to obtain the integral cosine image in the entire flipping process. Finally, the image is compared with the theoretical cosine image. If there is a significant difference, it is necessary to consider how to compensate; if the difference is small, the influence of the micromirror flipping on the measurement result can be ignored.

[0082] At the same time, in order to achieve higher-speed cosine fringe projection, the digital micromirror array can be windowed in actual operation. Taking Texas Instruments' DLP9500 chip as an example, in full-frame mode, its maximum operating frame rate is 22kfps, and through windowing, the maximum frame rate can be increased to 60kfps. However, the specific window size needs to be adjusted according to the undulations on the surface of the object being measured. Usually, when the surface undulations of the object being measured are not large, the window can be set to a quarter of the size of the target surface, at which point the digital micromirror array can reach its maximum frame rate of 60kfps. The relationship between the undulation degree of the object surface and the window size can be determined through geometric optical simulation to avoid the loss of three-dimensional morphology data during the measurement process due to inappropriate window size.

[0083] 2. Fringe image phase resolution algorithm with high speed and high noise resistance

[0084] In structured light measurement technology, in order to measure the three-dimensional shape of an object, phase information is extracted from the fringe image projected on the surface of the object through phase measurement methods such as three-step phase shift or four-step phase shift. However, the phase extracted from the fringe image is not the actual surface phase information of the object, but rather Figure 4 The wrapped phase is shown in (a) in Figure 1. The wrapped phase is limited to -π to +π and exhibits periodic fluctuations, which makes it impossible to directly reflect the true phase information of the object surface. Figure 4 The process of obtaining the continuous true phase shown in (b) is called phase resolution, which is the core step in stripe structured light measurement.

[0085] Traditional phase solution methods mainly include path method and optimization method. The path method completes phase recovery by accumulating phase difference along a specific path (such as the path with the smallest gradient or the highest quality), and the optimization method finds a solution that makes the phase diagram smooth by minimizing the residual of the phase difference. However, these two methods have high computational complexity and slow speed, which cannot meet the needs of fast solution. They are also sensitive to noise and cannot achieve correct phase solution in high-noise environments such as uneven optical imaging and background light interference. Therefore, traditional phase solution methods are not suitable for the high-speed and high-noise measurement scenarios of this method.

[0086] In order to meet the requirements of this method for high-speed and high-noise performance phase solution, this embodiment proposes phase solution based on deep neural network. This solution converts the traditional phase solution, an ill-posed problem, into a semantic segmentation problem in the field of deep learning, that is, the classification problem of image pixels. Figure 4 As shown in (c) in the figure, with the help of a deep convolutional neural network (DCNN), the package count corresponding to each pixel in the stripe image is classified and predicted, and then the package count is multiplied by 2π and added to the package phase to obtain the continuous true phase.

[0087] This method not only simplifies the traditional complex phase solution steps, but also uses the deep neural network to automatically learn and extract high-level features from the original image, effectively filtering out noise and background interference, and achieving higher solution accuracy and robustness. At the same time, the highly parallel nature of neural network calculations enables efficient parallel processing of phase solutions on GPU hardware accelerators, greatly improving the solution speed and meeting the strict real-time requirements of this method application.

[0088] In order to ensure the phase resolution performance of the network, this embodiment also performs network structure optimization and loss function design for the phase resolution scenario. First, by introducing the attention mechanism, the network is helped to automatically pay attention to the boundary area information of the stripe phase period during the training process, so as to achieve accurate classification of the package count and improve the accuracy of phase resolution. The attention mechanism can make the network more sensitive to key phase areas in complex backgrounds or high-noise environments, which helps to improve the robustness of the solution to high-noise data. In addition, combined with the gradient consistency characteristics between the true phase and the package phase in the phase solution, an optimized loss function including gradient loss is designed. Gradient loss can help the network learn the continuity characteristics of the phase map, reduce the common jumps and mutations in phase resolution, ensure that the phase resolution results are smooth and accurate, and further improve the performance of the model in complex environments.

[0089] In addition, in order to further speed up the network solution speed and improve the real-time response capability of the system, this embodiment is also based on the network acceleration research of TensorRT reasoning acceleration engine and numerical quantization. By using TensorRT, we fuse the convolution layer in the network with the batch normalization layer, reduce the number of redundant operations in the model, and achieve lightweight compression of the network structure, which not only reduces the computational burden, but also facilitates subsequent actual deployment. In order to further reduce the consumption of computing and memory resources, this embodiment will convert the model parameters and activation values ​​from FP32 type to INT8 type through numerical quantization, reduce the data bit width, and reduce memory requirements. This can not only greatly reduce memory usage, but also speed up the transmission speed of data between video memory and computing units, thereby giving full play to the advantages of GPU hardware in mixed precision operations. In the efficient INT8 reasoning mode, the running speed of the model will be significantly improved, ensuring that the network can achieve fast response without sacrificing the accuracy of phase separation, providing support for the high-speed and high-precision measurement of this method.

[0090] 3. Internal and external parameter calibration technology of high-speed digital projection system and line array camera

[0091] This embodiment intends to build Figure 5The stripe structured light high-speed three-dimensional shape measurement system shown in the figure. The system mainly includes key components such as line array camera, high-speed digital projection system, and high-speed translation stage. The object to be measured is placed on the high-speed translation stage. The high-speed digital projection system projects the cosine fringe image onto the surface of the object to be measured. The line array camera is used to collect image information on a certain contour line on the surface of the object to be measured. The projection lens group of the high-speed digital projection system consists of a tube lens, an objective lens, and a filter aperture. In order to ensure that the projected cosine fringes are clear on the surface of the object to be measured, the surface position of the digital micromirror array in the high-speed projection system needs to be optimized so that its imaging surface is parallel to the translation stage surface.

[0092] Before actual measurement, the parameters of each module of the system need to be calibrated. At this time, a checkerboard calibration plate is first placed on the high-speed translation stage. After the checkerboard calibration plate is quickly moved, the linear array camera is used to capture an image of the checkerboard calibration plate row by row. The internal and external parameter values ​​of the linear array camera and its imaging lens group are obtained through the image and the geometric optical transformation relationship. Then, a set of binary images for calibration are projected through the high-speed digital projection system, the checkerboard calibration plate is tilted at different angles, and a series of different projection images are captured by the linear array camera, and then the internal and external parameter values ​​of the high-speed digital projection system are obtained by processing these images.

[0093] Finally, standard samples with different surface optical properties will be made, including standard hemispheres, standard cubes, standard triangular pyramids, etc. The developed system will be used to perform three-dimensional morphology measurements on these shapes and obtain measurement results. These measurement results will be compared with the design values ​​to obtain the difference between them, thereby verifying the measurement accuracy and robustness of the system.

[0094] Finally, the main technical indicators achieved by this solution are as follows:

[0095] Through the high-speed digital projection system, the cosine fringe image switching rate can reach 60,000 frames per second; when the object moves at a speed of 1.5m / s, the lateral resolution of the measurement system can reach 0.1mm, the line scanning width is 190mm, and the height measurement accuracy reaches ±10μm.

[0096] This embodiment also provides a three-dimensional shape measurement device based on time-space staggered acquisition of structured light, including:

[0097] A translation stage, used to carry the object to be measured;

[0098] A digital projection system is used to project the structured light image onto the surface of the object to be measured to form an envelope structured light image;

[0099] Linear array camera, used to collect image information on the envelope line corresponding to the surface of the object being measured;

[0100] The controller is connected to the digital projection system and the linear array camera respectively, and is used to execute the above-mentioned three-dimensional morphology measurement method based on the spatiotemporal staggered acquisition of structured light.

[0101] It should be noted that the specific content and beneficial effects of the device of the present application can be found in the above-mentioned method embodiment, which will not be repeated here.

[0102] The preferred specific embodiments of the present invention are described in detail above. It should be understood that a person skilled in the art can make many modifications and changes based on the concept of the present invention without creative work. Therefore, any technical solution that can be obtained by a person skilled in the art through logical analysis, reasoning or limited experiments based on the concept of the present invention on the basis of the prior art should be within the scope of protection determined by the claims.

Claims

1. A three-dimensional shape measurement method based on spatiotemporal staggered acquisition of structured light, characterized in that: The following steps are involved: The object to be measured is placed on a horizontally movable translation stage, and the structured light image is projected onto the surface of the object to be measured through a digital projection system to form an envelope structured light image; the image information of the corresponding position on the surface of the object to be measured is collected through a line array camera; The digital projection system needs to project structured light images containing N images, namely S1, S2, …, S N When the object to be measured moves along with the translation stage, each time it moves a first distance, the digital projection system is controlled to cyclically switch the N structured light images S1, S2, ..., S N , and synchronously trigger the linear array camera to collect image information; The images collected by the linear array camera are recombined to form projected structured light images S1, S2, ..., S N The corresponding N cameras take structured light images I1, I2, ..., I N , the camera takes structured light images I1, I2, ..., I N The solution is performed and combined with ray tracing technology, the position information of each point on the surface of the measured object is decoded to obtain the three-dimensional shape measurement result of the measured object.

2. The three-dimensional shape measurement method based on time-space staggered acquisition of structured light according to claim 1, characterized in that: The projected structured light image is a cosine fringe structured light image. The N cameras take structured light images I1, I2, ..., I N The specific process of solving is as follows: According to the N cameras taking structured light images I1, I2, ..., I N Performing a simultaneous solution to obtain a wrapped phase, and using an unwrapping algorithm to unfold the wrapped phase into a continuous phase distribution, so as to correspond the pixels of the linear array camera with the pixels of the digital projection system, and further determine the three-dimensional coordinates of each pixel corresponding to the surface of the object under test; Sequentially perform corresponding processing of the three-dimensional coordinates of the surface of the object to be measured on different pixels of the image information collected by the linear array camera, and obtain the three-dimensional coordinate information of each point on the surface of the object along the envelope line corresponding to the image information; According to the image information on different envelope lines, the corresponding three-dimensional coordinate information of each point is obtained respectively, and combined to reconstruct the complete three-dimensional shape information of the measured object.

3. The three-dimensional shape measurement method based on spatiotemporal staggered acquisition of structured light according to claim 2, characterized in that: The unwrapping algorithm is a phase solution method based on a deep convolutional neural network, and the specific process includes: A deep convolutional neural network is used to capture the structured light images I1, I2, ..., I N The package count corresponding to each pixel in is classified and predicted, and then the package count is multiplied by 2π and added to the package phase to obtain the continuous phase distribution.

4. The three-dimensional shape measurement method based on spatiotemporal staggered acquisition of structured light according to claim 1, characterized in that: The projected structured light image is a Gray code structured light image. The N cameras take structured light images I1, I2, ..., I N The calculation process is to calculate the captured Gray code image, and then combine the ray tracing technology to decode the position information of each point on the surface of the object to be measured, and obtain the three-dimensional shape measurement result of the object to be measured.

5. The three-dimensional shape measurement method based on time-space staggered acquisition of structured light according to claim 1, characterized in that: The ray tracing technology is specifically: According to the N cameras taking structured light images I1, I2, ..., I N The acquisition time corresponding to each line array image in the image is obtained, and the ray tracing path collected by each pixel of the line array camera under different acquisition times is obtained to assist in solving the position information of each point on the surface of the measured object.

6. The three-dimensional shape measurement method based on spatiotemporal staggered acquisition of structured light according to claim 2, characterized in that: The digital projection system comprises: Digital micromirror array, used to control the state of light and generate binary images; A beam projection system, used for adjusting the binary image into a cosine fringe image; The light beam projection system comprises a light source, a collimator and a projection lens group. The light beam emitted by the light source is collimated into parallel light by the collimator and irradiated onto the surface of the digital micromirror array. After being reflected by the digital micromirror array, the light beam enters the projection lens group to form a cosine fringe image. The projection lens group comprises a tube lens, a filter aperture and an objective lens which are sequentially distributed. The filter aperture is located at the focal plane of the tube lens and the objective lens and is used for frequency domain filtering of the light field modulated by the digital micromirror array.

7. The three-dimensional shape measurement method based on time-space staggered acquisition of structured light according to claim 1, characterized in that: The digital projection system comprises: Liquid crystal spatial light controller, used to control the state of light and generate structured light images; A beam projection system, used to project a structured light image onto the surface of the object being measured; The light beam projection system includes a light source, a collimator and a projection lens group. The light beam emitted by the light source is collimated into parallel light by the collimator and irradiated onto the surface of the liquid crystal spatial light manipulator. After being reflected by the liquid crystal spatial light manipulator, the light beam enters the projection lens group to form a structured light image.

8. The three-dimensional shape measurement method based on spatiotemporal staggered acquisition of structured light according to claim 1, characterized in that: Before the method performs three-dimensional shape measurement on the object to be measured, the method also includes calibrating parameters of the three-dimensional shape measurement system composed of the translation stage, the digital projection system and the linear array camera; The parameter calibration process is specifically as follows: A checkerboard calibration plate is placed on the translation stage, the checkerboard calibration plate is tilted at different angles, and after the checkerboard calibration plate is driven to move by the translation stage, a line array camera is used to photograph row by row to obtain a set of calibration images of the checkerboard calibration plate; and internal and external parameter values ​​of the line array camera and its imaging lens group are obtained through the calibration image and the geometric optical transformation relationship; A set of binary images for calibration is projected through a digital projection system, the checkerboard calibration plate is tilted at different angles, and a series of different projection images are obtained by taking pictures with a line array camera, so as to calculate the internal and external parameter values ​​of the digital projection system.

9. The three-dimensional shape measurement method based on spatiotemporal staggered acquisition of structured light according to claim 7, characterized in that: The parameter calibration process also includes: Standard samples with different surface optical properties are manufactured, and the three-dimensional shape measurement system is used to perform three-dimensional shape measurement on the standard samples to obtain measurement results, which are compared with the corresponding design values ​​of the standard samples, thereby verifying the measurement accuracy and robustness of the three-dimensional shape measurement system.

10. A three-dimensional shape measurement device based on time-space staggered acquisition of structured light, characterized in that: include: A translation stage, used to carry the object to be measured; A digital projection system is used to project the structured light image onto the surface of the object to be measured to form an envelope structured light image; Linear array camera, used to collect image information on the envelope line corresponding to the surface of the object being measured; The controller is connected to the digital projection system and the linear array camera respectively, and is used to execute the three-dimensional morphology measurement method based on spatiotemporal staggered acquisition of structured light as described in any one of claims 1 to 9.

Images