Three-dimensional shape measurement method, measurement system and method for establishing depth information calibration table

Abstract

The present invention provides a method for measuring three-dimensional morphology, comprising: S11: projecting a plurality of randomly distributed speckle patterns onto a measured object; S12: sequentially collecting a plurality of frames of deformed speckle patterns on the measured object; S13: determining the intensity time series variation sequence of each spatial point on the measured object in the plurality of frames of deformed speckle patterns; S14: performing correlation operation on the intensity time series variation sequence of each spatial point and the intensity time series variation distribution in a depth information calibration table, wherein the depth information calibration table is constructed according to the projection patterns of the plurality of speckle patterns on a plurality of different depth planes; and S15: determining a matching point with the highest correlation coefficient with each spatial point, and constructing the three-dimensional morphology of the measured object based on the depth of each matching point. The present invention performs correlation operation on each spatial point on the measured object in time series, effectively retains the spatial high-frequency information of the measured object, and is suitable for measuring complex objects; at the same time, a high-speed mechanical rotating speckle projection system can be used to achieve low-cost rapid three-dimensional reconstruction.

Description

Technical Field

[0001] The present disclosure relates to the technical field of three-dimensional reconstruction, and in particular to a three-dimensional shape measurement method, a method for establishing a depth information calibration table, a three-dimensional shape measurement system, and a computer-readable storage medium. Background Art

[0002] The three-dimensional morphology reconstruction method based on digital speckle correlation can solve the problems of traditional stereo matching methods, such as the lack of reference information such as texture details and periodic repetitive features on the measured surface, which leads to difficulties in stereo matching, a large number of mismatches, and the inability to correctly reconstruct the three-dimensional surface shape of the measured object, thereby further expanding the scope of application of such methods.

[0003] The accuracy of speckle correlation calculation is the core factor that determines the accuracy of the measurement system. At present, the commonly used speckle correlation methods are mainly divided into two categories: spatial correlation and temporal correlation. Among them, spatial correlation uses two left and right cameras to simultaneously shoot the deformed speckle pattern, and then uses correlation operations to match the sub-areas in the left and right speckle images in the space to find matching points. However, this method has low accuracy when the measured object is relatively complex. This is because the window size of the spatial correlation is difficult to select automatically. For different measured objects, different correlation sub-window sizes will have a greater impact on the correlation accuracy. If the correlation sub-window is selected too large, the correlation peak will be smooth. If it is selected too small, there will be more pseudo peaks, which is not conducive to peak positioning. This makes it difficult for the accuracy of the spatial digital speckle correlation method to meet the needs of certain complex objects and high-precision three-dimensional surface measurement, which greatly limits the scope of application of the digital speckle correlation method.

[0004] Therefore, in order to avoid the problems caused by the selection of the size of the correlation sub-window in the digital speckle spatial correlation method, some scholars have proposed to perform correlation operations in time for stereo matching. The digital speckle time correlation method directly performs correlation operations on the time axis to obtain matching points, which improves the accuracy of stereo matching. However, at present, such methods need to construct a fast-changing speckle projection pattern in time sequence, and also require the left and right cameras to complete the matching of corresponding spatial points. The increase in projection speed and the increase in the number of cameras will greatly increase the cost of the measurement system.

[0005] The contents of the background technology section merely disclose the technologies known to the inventors and do not necessarily represent the prior art in the field. Summary of the invention

[0006] In view of one or more defects of the prior art, the present invention provides a method for measuring three-dimensional topography, comprising:

[0007] S11: Projecting multiple randomly distributed speckle patterns onto the object under test;

[0008] S12: sequentially collecting multiple frames of deformation speckle patterns on the object under test;

[0009] S13: determining a temporal variation sequence of intensity of each spatial point on the measured object in the multi-frame deformed speckle pattern;

[0010] S14: performing a correlation operation on the intensity time-series variation sequence of each spatial point and the intensity time-series variation distribution in the depth information calibration table, wherein the depth information calibration table is constructed according to the projection patterns of the multiple speckle patterns on multiple different depth planes; and

[0011] S15: Determine a matching point with the highest correlation coefficient with each of the spatial points, and construct a three-dimensional shape of the object to be measured based on the depth of each matching point.

[0012] According to one aspect of the present invention, step S11 comprises: projecting a plurality of randomly distributed speckle patterns onto the object to be measured by a rotating projection device, wherein the rotating projection device comprises a light source, a speckle disk, a motor and a projection lens, wherein the motor drives the speckle disk to rotate, and the light source irradiates a light beam to different positions on the speckle disk, and projects a plurality of speckle patterns onto the object to be measured through the projection lens.

[0013] According to one aspect of the present invention, step S12 includes: sequentially collecting multiple frames of deformed speckle patterns on the measured object by only one camera.

[0014] According to one aspect of the present invention, the depth information calibration table is constructed in the following manner:

[0015] S21: Projecting a randomly distributed speckle pattern onto a standard plane;

[0016] S22: sequentially collecting multiple frames of deformed speckle patterns on the standard plane;

[0017] S23: moving the standard plane by a first distance in the depth direction, and sequentially collecting multiple frames of deformed speckle patterns on the standard plane;

[0018] S24: repeat step S23 until the standard plane is moved to a preset depth;

[0019] S25: Based on the multi-frame deformed speckle patterns of multiple depth planes, a depth information calibration table of the intensity time-series change distribution of each point in space is established.

[0020] According to one aspect of the present invention, the speckle pattern is a speckle field that randomly combines local and global positions.

[0021] According to one aspect of the present invention, step S12 further comprises: determining a time sequence correspondence or a corresponding frame start position between the acquired multiple frames of deformed speckle patterns and the projected speckle pattern.

[0022] According to one aspect of the present invention, the intensity temporal variation sequence includes a sequence of grayscale values ​​of pixel points corresponding to each spatial point on the measured object in each frame of the deformed speckle pattern.

[0023] According to one aspect of the present invention, step S14 includes: determining the grayscale value of the pixel point that changes in the time series in the multiple frames of the deformed speckle pattern according to the grayscale value of the pixel point corresponding to each spatial point on the measured object in each frame of the deformed speckle pattern; and performing correlation operation on the grayscale value of the pixel point corresponding to each spatial point and the grayscale value that changes in the time series with the grayscale value of each pixel point that has a corresponding relationship in the depth information calibration table and the grayscale value that changes in the time series.

[0024] According to one aspect of the present invention, step S15 includes: determining the depth plane where the matching point with the highest temporal correlation coefficient with each spatial point is located, and constructing the three-dimensional shape of the object under test based on the depth information of each depth plane.

[0025] The present invention also provides a method for establishing a depth information calibration table, comprising:

[0026] S21: Projecting a randomly distributed speckle pattern onto a standard plane;

[0027] S22: sequentially collecting multiple frames of deformed speckle patterns on the standard plane;

[0028] S23: moving the standard plane in the depth direction by a first distance, and sequentially collecting multiple frames of deformed speckle patterns on the standard plane;

[0029] S24: repeating step S23 until the standard plane is moved to a preset depth; and

[0030] S25: Based on the multi-frame deformed speckle patterns of the multiple depth planes, a depth information calibration table of the intensity time-series change distribution of each pixel point is established.

[0031] According to one aspect of the present invention, step S21 comprises: projecting a plurality of randomly distributed speckle patterns onto the object to be measured by a rotating projection device, wherein the rotating projection device comprises a light source, a speckle disk and a motor, wherein the motor drives the speckle disk to rotate, and the light source irradiates a light beam to different positions on the speckle disk to project a plurality of speckle patterns onto the object to be measured.

[0032] According to one aspect of the present invention, step S22 includes: sequentially collecting multiple frames of deformed speckle patterns on the measured object by only one camera.

[0033] According to one aspect of the present invention, the speckle pattern is a combination of local and global position random speckle field.

[0034] According to one aspect of the present invention, step S22 further comprises: determining a time sequence correspondence or a corresponding frame start position between the acquired multiple frames of deformed speckle patterns and the projected speckle pattern.

[0035] According to one aspect of the present invention, the depth information calibration table includes the grayscale values ​​of the pixels corresponding to each point in space on the standard plane in each frame of the deformed speckle pattern on multiple depth planes, and the grayscale values ​​of the pixels changing in the time series in the multiple frames of the deformed speckle pattern.

[0036] The present invention also provides a three-dimensional shape measurement system, comprising:

[0037] a rotating projection device configured to project a randomly distributed speckle pattern;

[0038] an image acquisition device configured to acquire a pattern; and

[0039] The control device is configured to control the rotation projection device and the image acquisition device to jointly execute the measurement method as described above to construct the three-dimensional shape of the measured object.

[0040] According to one aspect of the present invention, the image acquisition device includes only one camera.

[0041] According to one aspect of the present invention, the randomly distributed speckle pattern is a speckle field randomly combined with local and global positions.

[0042] According to one aspect of the present invention, it also includes:

[0043] Standard plane;

[0044] A mobile platform, used to carry the standard plane and configured to drive the standard plane to move in the depth direction;

[0045] Wherein, the control device is further configured to control the rotating projection device, the image acquisition device and the mobile platform to jointly execute the method as described above to establish a depth information calibration table.

[0046] According to one aspect of the present invention, the rotary projection device comprises:

[0047] light source; and

[0048] The speckle disk is disposed downstream of the optical path of the light source and is configured to modulate the light emitted by the light source into a randomly distributed speckle pattern.

[0049] According to one aspect of the present invention, the rotary projection device further comprises:

[0050] The motor is configured to drive the speckle disk to rotate.

[0051] According to one aspect of the present invention, the rotary projection device further comprises:

[0052] A projection lens is disposed downstream of the optical path of the speckle disk and is configured to project the randomly distributed speckle pattern.

[0053] According to one aspect of the present invention, the rotary projection device further comprises:

[0054] The collimating device is disposed between the light source and the speckle disk, and is configured to collimate the light emitted by the light source and then make it incident on the speckle disk.

[0055] According to one aspect of the present invention, a grating encoder is provided on the inner circumference and / or the outer circumference of the speckle disk.

[0056] According to one aspect of the present invention, it also includes:

[0057] a detection device configured to obtain a rotational position of the speckle disk based on the grating encoder;

[0058] The control device is further configured to determine the time sequence correspondence or the corresponding frame start position between the acquired multiple frames of deformed speckle patterns and the projected speckle pattern according to the rotation position acquired by the detection device.

[0059] The present invention also provides a computer-readable storage medium, comprising computer-executable instructions stored thereon, wherein the executable instructions implement the above-mentioned method and the above-mentioned measurement method when executed by a processor.

[0060] The present invention proposes a new method and system for fast three-dimensional morphology measurement based on time-series speckle correlation operation. A low-cost, high-speed switching, time-series changing speckle pattern projection is constructed by using a mechanical rotating projection device, and only one camera is placed at another angle to collect the corresponding speckle deformation pattern. Then, a high-precision translation stage is used to drive the standard plane to move in the depth direction, and multiple rotating switching speckle patterns are photographed on each plane, and a point-to-point lookup table corresponding to the speckle pattern and depth information in the time sequence is established. Finally, when measuring the object to be measured, each point in space performs an independent correlation operation along the time axis, and the known depth plane with the highest correlation coefficient is found to determine the depth of the spatial point. This method performs correlation operations on each spatial point in the time sequence, which can effectively retain the spatial high-frequency information of the object to be measured and is suitable for the measurement of complex objects. At the same time, the use of a high-speed mechanical rotating speckle projection system can achieve low-cost fast three-dimensional reconstruction. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] The drawings constituting a part of the present disclosure are used to provide a further understanding of the present disclosure. The illustrative embodiments of the present disclosure and their descriptions are used to explain the present disclosure and do not constitute an improper limitation on the present disclosure. In the drawings:

[0062] Figure 1 A flow chart of a method for measuring three-dimensional topography according to an embodiment of the present invention is shown;

[0063] Figure 2 A schematic diagram of a three-dimensional shape measurement system according to an embodiment of the present invention is shown;

[0064] Figure 3 A schematic diagram of a deep matching search process according to an embodiment of the present invention is shown;

[0065] Figure 4 A flow chart showing a method for establishing a depth information calibration table according to an embodiment of the present invention is shown;

[0066] Figure 5 Schematic diagrams of four projected speckle fields according to an embodiment of the present invention are shown;

[0067] Figure 6 shows a comparison of three random speckle fields taken by the same system according to an embodiment of the present invention;

[0068] Figure 7 shows the deformed speckle images taken at different times according to one embodiment of the present invention;

[0069] Figure 8 A three-dimensional shape reconstruction effect diagram of an embodiment of the present invention is shown;

[0070] Fig. 9 A module diagram of a measurement system according to an embodiment of the present invention is shown;

[0071] Fig.10 A module diagram of a rotary projection device according to an embodiment of the present invention is shown;

[0072] Fig.11 A schematic diagram of an optical path of a rotating projection device according to an embodiment of the present invention is shown;

[0073] Fig.12 A schematic diagram of a speckle disk with a grating encoder according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0074] In the following, only some exemplary embodiments are briefly described. As those skilled in the art will appreciate, the described embodiments may be modified in various ways without departing from the spirit or scope of the present invention. Therefore, the drawings and descriptions are considered to be exemplary and non-restrictive in nature.

[0075] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, which are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first" and "second" are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.

[0076] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection, an electrical connection, or can communicate with each other; it can be directly connected, or indirectly connected through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0077] In the present invention, unless otherwise clearly specified and limited, a first feature being "above" or "below" a second feature may include that the first and second features are in direct contact, or may include that the first and second features are not in direct contact but are in contact through another feature between them. Moreover, a first feature being "above", "above" and "above" a second feature includes that the first feature is directly above and obliquely above the second feature, or simply means that the first feature is higher in level than the second feature. A first feature being "below", "below" and "below" a second feature includes that the first feature is directly above and obliquely above the second feature, or simply means that the first feature is lower in level than the second feature.

[0078] The disclosure below provides many different embodiments or examples to realize different structures of the present invention. In order to simplify the disclosure of the present invention, the parts and settings of specific examples are described below. Of course, they are only examples, and the purpose is not to limit the present invention. In addition, the present invention can repeat reference numbers and / or reference letters in different examples, and this repetition is for the purpose of simplicity and clarity, which itself does not indicate the relationship between the various embodiments and / or settings discussed. In addition, the present invention provides various specific examples of processes and materials, but those of ordinary skill in the art can be aware of the application of other processes and / or the use of other materials.

[0079] The three-dimensional shape reconstruction method based on digital speckle correlation is an optical measurement method that uses the speckle grayscale value that carries the three-dimensional surface information of the object as a carrier, and determines the displacement and shape of the object by calculating the correlation of the speckle field before and after the deformation of the object. Its basic principle is to use the grayscale of the speckle pattern as an information carrier, and to determine the similarity of the sub-regions of the object before and after the deformation by calculating the correlation value of the sub-regions before and after the deformation of the object. It is believed that there is a corresponding relationship between the position of the sub-region after deformation with the maximum (or minimum) correlation value and the position of the sub-region before deformation, so that the deformation or displacement of the object can be determined.

[0080] The technical solution of the embodiment of the present invention is: at the projection end, a speckle disk made on a transparent disk is driven by a mechanical motor to rotate at high speed, thereby constructing a low-cost, high-speed switching speckle pattern. At the other end, a camera is placed to synchronously shoot the speckle pattern modulated by the measured object. After a series of speckle patterns are obtained, an independent correlation operation is performed on each point on the camera target surface on the time axis, and finally the real height distribution of each spatial point is determined by calculating the depth plane with the highest correlation with the speckle time series distribution on each depth plane established in advance.

[0081] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0082] Figure 1 A flow chart of a method for measuring a three-dimensional shape according to an embodiment of the present invention is shown. The measuring method 10 comprises:

[0083] In step S11: a plurality of randomly distributed speckle patterns are projected onto the object to be measured. The speckle patterns are, for example, pre-set, and the projection mode is, for example, continuous playback like a film or switching like a slide show.

[0084] According to a preferred embodiment of the present invention, a plurality of randomly distributed speckle patterns are projected onto the object to be measured by a rotating projection device, wherein the rotating projection device comprises a light source, a speckle disk, a motor and a projection lens, wherein the motor drives the speckle disk to rotate, and the light source irradiates a light beam to different positions on the speckle disk, and projects a plurality of speckle patterns onto the object to be measured through the projection lens. Figure 2 , a rotating projection device is arranged in front of the object to be measured, and a speckle pattern is provided on the speckle disk along its circumference. The speckle pattern is preferably composed of a light-transmitting part and a light-opaque part, and the light-transmitting part and the light-opaque part can be randomly distributed to form a speckle pattern. A light beam emitted by a light source (such as an optical machine) is projected onto a certain sector of the speckle disk, and a speckle pattern is generated after being modulated by the sector. The speckle disk rotates under the drive of a motor (not shown), and each sector modulates the light beam in turn to generate N speckle patterns. The N speckle patterns that change in time sequence are imaged by a projection lens (not shown) and are continuously projected onto the surface of the object to be measured. Among them, the object to be measured is located within a preset depth range, and the distance from the rotating projection device is to ensure that the speckle pattern projected by the rotating projection device covers the surface of the object to be measured.

[0085] In step S12: multiple frames of deformed speckle patterns on the measured object are collected in sequence. The projected N speckle patterns are reflected by the surface of the measured object to generate deformed speckle patterns. According to the order of the projected speckle patterns, M frames of deformed speckle patterns are collected in sequence, where M≤N, that is, the deformed speckle patterns on the measured object can be collected one by one corresponding to the speckle patterns projected by the rotating projection device, or only a few consecutive frames or a few interval frames can be collected in time sequence. The larger the value of M, the higher the measurement accuracy.

[0086] According to a preferred embodiment of the present invention, multiple frames of deformed speckle patterns on the object under test are collected sequentially by only one camera. Figure 2 , a camera is set in front of the object to be measured, the object to be measured is in the camera's field of view, and the camera can collect all the deformed speckle patterns on the object to be measured. The position of the camera and the rotating projection device is fixed.

[0087] In step S13: determine the intensity time-series variation sequence of each spatial point on the measured object in the multi-frame deformed speckle pattern. Specifically, in step S12, a camera is used to synchronously shoot the speckle pattern modulated by the measured object. After a series of deformed speckle patterns are obtained, independent operations are performed on the time axis for the pixel points corresponding to each spatial point on the camera target surface. Since the positions of the speckle disk, the measured object and the camera are all fixed during the image acquisition process, the corresponding pixel points on each image in the multi-frame image acquired by the camera belong to the same spatial point on the measured object in the physical space.

[0088] According to a preferred embodiment of the present invention, the intensity temporal variation sequence includes a sequence of grayscale values ​​of pixels corresponding to each spatial point on the measured object in each frame of the deformed speckle pattern. For example, the camera collects M frames of deformed speckle pattern, uses grayscale values ​​to characterize the intensity of pixels in the deformed speckle pattern, calculates the grayscale value of the pixel corresponding to each spatial point in each frame of the deformed speckle pattern, and the temporal variation of the grayscale value of the pixel in all M frames of the deformed speckle image. Alternatively, in step S13, some frames in the M frames of the deformed speckle pattern can be selected to determine the intensity temporal variation sequence of each spatial point, without selecting all M frames of the deformed speckle pattern.

[0089] In step S14: the intensity time series change sequence of each spatial point is correlated with the intensity time series change distribution in the depth information calibration table, wherein the depth information calibration table is constructed according to the projection patterns of multiple speckle patterns on multiple planes at different depths. In the present invention, the depth information calibration table uses the same multiple speckle patterns as step S11, and projects them on planes at different depths and collects patterns, so as to obtain the intensity time series change of each point in the space. For example, the projection plane is first fixed at the first depth (first distance), and multiple speckle patterns are sequentially projected onto the projection plane and the patterns are collected, so that the intensity time series change sequence of each pixel point in the image at the first depth is obtained. Then the projection plane is fixed at the second depth (second distance), and the above steps are repeated to obtain the intensity time series change sequence of each pixel point in the image at the second depth. Repeating the above steps, multiple intensity time series change sequences are obtained, corresponding to multiple depths respectively. Based on multiple intensity time series change sequences and multiple depths, the depth information calibration table can be constructed. In step S14, the depth information calibration table is used as a basis to calculate the correlation between the intensity time series variation sequence of each spatial point and the multiple intensity time series variation sequences in the depth information calibration table.

[0090] The preferred method for creating the depth information calibration table will be described later. In step S14, the speckle pattern modulated by the object to be measured is compared with the speckle pattern calibrated in the depth information calibration table to find the matching point of each spatial point on the object to be measured.

[0091] According to a preferred embodiment of the present invention, the grayscale value of each pixel corresponding to each spatial point on the measured object in each frame of the deformed speckle pattern is determined in a time series in multiple frames of the deformed speckle pattern; the grayscale value of the pixel corresponding to each spatial point and the grayscale value that changes in the time series are correlated with the grayscale value of each pixel corresponding to each spatial point and the grayscale value that changes in the time series in the depth information calibration table. Specifically, the rotating projection device projects the speckle pattern that changes with time onto the surface of the measured object, and the camera synchronously captures the deformed speckle pattern after being modulated by the measured object. Figure 3 As shown in the figure, the right side shows the deformation pattern on the measured object in M ​​frames, and the left side shows the speckle pattern on a certain depth plane in the depth information calibration table in N frames. For the convenience of description, it is assumed that M = N. Based on the camera coordinate system, the pixel point corresponding to a spatial point on the measured object in the deformation speckle pattern is p1(x1, y1), and the pixel point corresponding to the speckle pattern on a certain depth plane is p2(x2, y2). The correlation value C can be obtained by performing correlation operation in the time domain as follows:

[0092]

[0093] Among them, g1(x1,y1,t) represents the gray value of the pixel p1(x1,y1) in the t-th frame of the deformed speckle pattern collected by the camera, represents the time average gray value of the pixel p1(x1,y1) in all M frames of the deformed speckle pattern of the camera. g2(x2,y2,t,m) represents the gray value of the pixel p2(x2,y2) in the t-th frame speckle pattern collected by the camera in the x-th depth plane. Represents the temporal average grayscale value of the pixel point p2(x2,y2) in all N frames of speckle pattern of the camera in this depth plane. The correlation value between the point p2(x2,y2) and the point p1(x1,y1) in the x-th depth plane is calculated by formula (1). Traverse all depth planes and calculate the correlation value between the point p2(x2,y2) and the point p1(x1,y1) in each depth plane. Then, perform the above calculation on each spatial point on the measured object to independently obtain a set of correlation values ​​corresponding to each spatial point.

[0094] Preferably, the positions of pixel point p1 (x1, y1) and pixel point p2 (x2, y2) are both integer pixels, and sub-pixels can be obtained by interpolation or fitting. Sub-pixels are used to improve the accuracy of correspondence from spatial points to pixel points.

[0095] The above is a correlation operation between pixel points. Preferably, a correlation operation can also be performed between pixel regions. Figure 3 , the depth occupied by the M-frame deformed speckle pattern on the time axis is St, and the area Sx*Sy centered on the pixel point p1(x1,y1) is taken; the M-frame speckle pattern corresponding to the time depth is selected on the depth plane, and the area Sx*Sy centered on the pixel point p2(x2,y2) is taken. The correlation operation of the two areas can reduce the amount of calculation and speed up the measurement.

[0096] In step S15: determine the matching point with the highest correlation coefficient with each spatial point, and construct the three-dimensional shape of the object under test based on the depth of each matching point. The matching point search based on the correlation coefficient is performed by performing a correlation operation on the intensity time series change of each spatial point on the surface of the object under test and the existing intensity change distribution in the depth information calibration table, wherein the depth position of the matching point with the strongest correlation in the depth information calibration table is the depth position of the current spatial point, thereby obtaining the depth distribution of all spatial points.

[0097] According to a preferred embodiment of the present invention, the depth plane where the matching point with the highest time series correlation coefficient with each spatial point is located is determined, and the three-dimensional shape of the object to be measured is constructed based on the depth information of each depth plane. In step S14, the correlation value between the point p2 (x2, y2) and the point p1 (x1, y1) in each depth plane is obtained, where the p2 (x2, y2) corresponding to the largest correlation value is the matching point with the highest correlation coefficient, and the depth corresponding to the depth plane where it is located is the real depth of p1 (x1, y1). Then, based on a set of correlation values ​​corresponding to each spatial point, the depth information of the matching point corresponding to the largest correlation value is determined, thereby obtaining the depth information of each spatial point on the object to be measured, and finally constructing the three-dimensional shape of the object to be measured.

[0098] The three-dimensional shape measurement method 10 is introduced above through steps S11-S15. By actively projecting a speckle pattern onto the measured object, the speckle pattern is used to randomly encode the surface shape of the measured object in time for correlation matching, and the deformed speckle pattern modulated by the measured object and the corresponding matching point in the depth information calibration table are found, so as to obtain the depth distribution of the measured object, and low-cost rapid three-dimensional reconstruction is achieved by adopting a high-speed mechanical rotating speckle projection system and a camera.

[0099] According to a preferred embodiment of the present invention, referring to Figure 4 , construct the depth information calibration table through method 20:

[0100] In step S21: projecting a randomly distributed speckle pattern onto the standard plane. Preferably, a plurality of randomly distributed speckle patterns are projected onto the standard plane by rotating the projection device, referring to Figure 2, the object to be measured is replaced with a standard plane, the rotating projection device is set in front of the standard plane, the light beam emitted by the light source is projected onto a sector of the speckle disk, and a speckle pattern is generated after being modulated by the sector. The speckle disk rotates under the drive of the motor, and each sector modulates the light beam in turn to generate N speckle patterns. The N speckle patterns that change in time sequence are projected onto the standard plane after being imaged by the projection lens. The size of the standard plane or the distance from the rotating projection device is reasonably designed to ensure that the projected speckle pattern roughly covers the standard plane or is smaller than the area of ​​the standard plane, and then the position of the rotating projection device is fixed. Among them, the standard plane can be a plane commonly used during calibration, and the present invention does not limit the shape, material, reflectivity, etc. of the standard plane.

[0101] In step S22: sequentially collect multiple frames of deformed speckle patterns on the standard plane. The projected N speckle patterns form an amplified speckle pattern on the standard plane. According to a preferred embodiment of the present invention, only one camera is used to sequentially collect N frames of speckle patterns on the standard plane. Figure 2 ,Set the camera in front of the standard plane, the standard plane is within the camera field of view, the camera can collect all the speckle patterns on the standard plane, and fix the camera position.

[0102] In step S23: the standard plane is moved in the depth direction by a first distance, and multiple frames of deformed speckle patterns on the standard plane are sequentially collected. For example, the standard plane is driven to move in the depth direction by a high-precision translation stage, and N frames of speckle patterns are captured at each depth. The first distance is greater than or equal to the step distance of the high-precision translation stage, and the smaller the first distance, the higher the measurement accuracy.

[0103] In step S24: repeat step S23 until the standard plane is moved to the preset depth. In the three-dimensional shape measurement method 10, the depth of each spatial point on the measured object is determined as a relative depth. The position of the rotating projection device can be used as a reference point, or the position of the camera can be used as a reference point to move the standard plane within the preset depth range to measure a set (N frames) of speckle patterns corresponding to each depth information.

[0104] In step S25: based on the multi-frame deformed speckle pattern of multiple depth planes, a depth information calibration table of the intensity temporal variation distribution of each pixel is established. According to a preferred embodiment of the present invention, the intensity temporal variation distribution includes the gray value of each pixel in each frame of the speckle pattern collected by the camera and its temporal variation. For example, the gray value of each pixel in each frame of the deformed speckle pattern and the temporal average gray value of each pixel in all N frames of the deformed speckle pattern are calculated.

[0105] The method of establishing a depth information calibration table is introduced above through steps S21-S25. After the projection optical path is designed, it is necessary to first create a table for the speckle information of different depth planes before measurement. When creating the table, a precision translation stage is used to drive the standard plane to move, for example, at intervals of 0.1 mm (set according to actual needs, the higher the measurement accuracy requirement, the smaller the interval). On each depth plane, a rotating projection device is used to project the time-series changing speckle pattern to the standard plane, and a camera is used to synchronously shoot the deformed speckle pattern. After each depth plane collects the pattern projected by the speckle disk rotating one circle, it is sufficient. After performing the above operations on all depth planes within the measurement depth range, the speckle distribution of different sectors of the speckle disk on all different depth planes can be obtained. Therefore, a unique intensity distribution on the time axis can be established for each pixel point on the camera target surface at different depth planes. In subsequent measurements, for a certain pixel point, the depth distribution of each spatial point corresponding to it can be determined by finding the intensity distribution with the largest correlation coefficient with it in the depth direction, thereby quickly constructing the three-dimensional morphology of the object under test.

[0106] The method 20 for establishing a depth information calibration table and the method 10 for performing three-dimensional shape measurement based on the depth information calibration table are respectively introduced above. According to a preferred embodiment of the present invention, step S12 further includes: determining the time sequence correspondence between the acquired multiple frames of deformed speckle patterns and the projected speckle pattern or the corresponding frame start position. That is, the correspondence between the deformed speckle pattern on the measured object and the speckle pattern of each depth plane in the depth information calibration table on the time axis is established. For example, when the camera performs interval acquisition of the projected speckle pattern, it is necessary to determine the correspondence between the two on the time axis; or the first frame of the deformed speckle pattern is matched with the speckle pattern in the depth information calibration table, and then continuous acquisition is performed, so that each frame of the deformed speckle pattern can be matched with the speckle pattern of each depth plane in the depth information calibration table on the time axis. Although the intensity time sequence distribution of each spatial point is unique, it takes a lot of time to traverse the depth information calibration table to confirm the depth of the spatial point. The speckle patterns of different sectors are located. During measurement, the detection device can be used to obtain the rotational position information of the speckle disk, which provides reference information for subsequent speckle pattern matching, narrows the timing correlation range, and can effectively improve the correlation operation and reconstruction efficiency.

[0107] Those skilled in the art will understand that in order to ensure the accuracy of the measurement, the parameter settings and position of the rotation projection device, as well as the parameter settings and position of the camera when establishing the depth information calibration table, are preferably consistent with those when performing three-dimensional morphology measurement, and the object to be measured is placed within a preset depth range.

[0108] According to a preferred embodiment of the present invention, the projected speckle pattern is a speckle field that randomly combines local and global positions.

[0109] Projection digital speckle is mainly divided into position random distribution speckle field and gray random distribution speckle field. The gray random distribution speckle field is referred to as Figure 5 The middle (a) figure refers to the random distribution of the grayscale values ​​of scattered speckles. Taking an 8-bit image as an example, the grayscale values ​​are randomly distributed between 0 and 255, while the position-randomly distributed speckle refers to the random distribution of the positions of scattered speckles. Taking an 8-bit image as an example, the position-randomly distributed speckle field has only two grayscale values, 0 and 255, and the positions of scattered spots are random, so it is easier to achieve and prepare.

[0110] According to the type of random distribution area, the position random distribution speckle field is divided into three types: global position random distribution, local position random distribution, and local and global position random combination. Figure 5 In the middle (b), the position of the bright spot (the point with gray value of 255) is globally random. The local position is randomly distributed in the speckle field, refer to Figure 5 In Figure (c), the bright spot only appears randomly in a certain area (e.g., a 3*3 area), and there is only one speckle bright spot in each area. When the local positions of the speckle bright spots are randomly distributed, plus some random speckle bright spots in the global positions, a combination of local and global random speckle fields is formed. Figure 5 As shown in Figure (d) in the figure, the number of random speckle bright spots in the global position can be controlled by the proportion in the whole image, thereby adjusting the density of the speckle field. According to a preferred embodiment of the present invention, the proportion of speckle bright spots in the speckle field combined with the local and global positions is close to 50%.

[0111] Although the grayscale random distribution speckle field has good randomness, it is limited by the grayscale dynamic range of the projector and CCD resolution, resulting in the grayscale random distribution speckle field being greatly affected by noise and shadows. The global position random speckle field may cause some local positions to have poor randomness. The randomness and correlation of the local position random and the combination of local and global position random speckle fields are relatively good, and the ability to resist shadows and noise is stronger. The average grayscale gradient parameter is effective in evaluating the quality of the speckle pattern. A good speckle pattern should have a large average grayscale gradient value.

[0112] Figure 6 The average grayscale gradient values ​​of the local and global position random combination, local position random and global position random speckle fields actually captured by the same imaging system are 23.64, 19.44 and 17.11 respectively. It can be seen from the experimental results that the average grayscale gradient value of the local and global position random combination speckle field is larger than the average grayscale gradient value of the regional random distribution speckle field, so a preferred embodiment of the present invention adopts a projection speckle pattern that is a local and global position random combination speckle field.

[0113] In the above, various randomly distributed speckle fields are compared through experiments. In the following, three-dimensional morphology measurement is performed based on the random combination of speckle fields at the preferred local and global positions.

[0114] First, create a depth information calibration table using method 10. Figure 2 , keep the parameters and positions of the rotating projection device and the camera unchanged to reduce the influence of other factors on the speckle matching, then place the object to be measured within the depth measurement range, and after the camera captures the first frame of the deformed speckle pattern, for example, use the phase mark information to determine the sector of the speckle disk, thereby confirming the starting point of the time-related mark. Then, control the speckle disk to capture a deformed speckle pattern every time it rotates a preset angle (for example, 5°), or capture a deformed speckle pattern every preset time interval, for example, a total of 12 images are captured. Figure 7 Figure (a) shows the first deformed speckle pattern. Figure 7 FIG. (b) shows the twelfth deformed speckle pattern. According to the measurement method 10, the depth information corresponding to each spatial point is determined, and the reconstruction result of the three-dimensional morphology of the measured object is as follows: Figure 8 As shown in the figure, it can be seen from the results that the detail information of the complex petal model can be well reconstructed by utilizing the temporal correlation, and the measurement of the complex surface can be completed.

[0115] The present invention also designs a three-dimensional shape measurement system, referring to Fig. 9 The measuring system 100 includes a rotating projection device 101, an image acquisition device 102 and a control device 103. The rotating projection device 101 is configured to project a randomly distributed speckle pattern. According to a preferred embodiment of the present invention, the randomly distributed speckle pattern projected by the rotating projection device 101 is a speckle field randomly combined with local and global positions. The image acquisition device 102 is configured to acquire a pattern. According to a preferred embodiment of the present invention, the image acquisition device includes only one camera. The control device 103 is configured to control the rotating projection device 101 and the image acquisition device 102 to jointly perform the measuring method 10 as described above to construct the three-dimensional morphology of the object to be measured.

[0116] According to a preferred embodiment of the present invention, the measurement system 100 also includes a standard plane 104 and a mobile platform 105. The standard plane 104 is a common plane in the calibration process. The mobile platform 105 is used to carry the standard plane 104 and is configured to drive the standard plane 104 to move in the depth direction. Preferably, the mobile platform 105 is a precision translation stage, for example, it can drive the standard plane 104 to move at intervals of 0.1 mm. The control device 103 is also configured to control the rotation projection device 101, the image acquisition device 102 and the mobile platform 105 to jointly execute the method 20 as described above to establish a depth information calibration table. That is, the standard plane 104 and the mobile platform 105 are needed when establishing the depth information calibration table. When actually measuring the three-dimensional morphology of the object, the position and parameter settings of the rotation projection device 101 and the image acquisition device 102 are kept unchanged, and the standard plane 104 and the mobile platform 105 are replaced by the object to be measured.

[0117] According to a preferred embodiment of the present invention, referring to Fig.10 The rotating projection device 101 includes a light source 1011 and a speckle disk 1012, wherein the light source 1011, for example, an optical machine, emits a light beam, and the speckle disk 1012, for example, is a customized coded speckle disk, which is arranged downstream of the light path of the light source 1011 and is configured to modulate the light emitted by the light source into a randomly distributed speckle pattern.

[0118] According to a preferred embodiment of the present invention, the rotating projection device 101 further includes a motor 1013 configured to drive the speckle disk 1012 to rotate.

[0119] According to a preferred embodiment of the present invention, the rotating projection device 101 further comprises a projection lens 1014, which is disposed downstream of the optical path of the speckle disk 1012 and is configured to project the randomly distributed speckle pattern.

[0120] According to a preferred embodiment of the present invention, the rotary projection device 101 further includes: a collimating device 1015 disposed between the light source 1011 and the speckle disk 1012 , configured to collimate the light emitted by the light source and then make it incident on the speckle disk 1012 .

[0121] The optical path of the rotating projection device 101 is set as follows Fig.11As shown, the light emitted by the light source 1011 is evenly collimated by the collimator 1015 and then projected onto the speckle disk 1012. The speckle disk 1012 modulates the light beam to produce a randomly distributed speckle pattern. When the speckle disk 1012 is in a stationary state, the sector of the disk currently illuminated is imaged by the projection lens 1014, and a single-frame speckle projection is realized. When the disk rotates under the drive of the motor 1013, the speckle pattern that changes in time sequence will be continuously projected onto the surface of the object to be measured through the projection lens 1014, and high-speed speckle switching projection is realized. For the customized speckle disk 1012, by adjusting the rotation speed of the motor 1013 and the imaging recording speed of the image acquisition device 102 (such as a camera), a single-frame recording mode or a multi-frame speckle recording mode can be flexibly realized.

[0122] According to a preferred embodiment of the present invention, the speckle disk 1012 is provided with a grating encoder on the inner periphery and / or the outer periphery. Fig.12 1 and 10 respectively show a speckle disk 1012 with a grating encoder disposed on the inner periphery and a speckle disk 1012 with a grating encoder disposed on the outer periphery. The establishment of the point-to-point depth information calibration table is to determine the starting position of the temporal speckle pattern through the grating encoding information on the outer periphery or inner periphery of the speckle disk, and to construct a mapping relationship between the temporal speckle distribution and the depth of each point by rotating the disk in different known depth planes.

[0123] According to a preferred embodiment of the present invention, the measurement system 100 further includes a detection device 1016 configured to obtain the rotation position of the speckle disk 1012 based on the grating encoder. The control device 103 is further configured to determine the time sequence correspondence between the collected multiple frames of deformed speckle patterns and the projected speckle pattern or the corresponding frame start position according to the rotation position obtained by the detection device 1016.

[0124] Because the present invention adopts a time-dependent decoding strategy, it is necessary to construct a time-varying speckle distribution through the high-speed rotation of the speckle disk 1012. In order to roughly locate the speckle areas of different sectors, a circle of grating encoders is designed on the outer circumference or inner circumference of the speckle disk 1012 to mark each sector. When the system is implemented, the detection device 1016 is used to obtain the disk rotation position information to provide initial information for subsequent speckle matching, reduce the time series correlation range, and effectively improve the correlation operation and reconstruction efficiency.

[0125] The present invention also relates to a computer-readable storage medium, comprising computer-executable instructions stored thereon, wherein the executable instructions implement the method 20 as described above and the measuring method 10 as described above when executed by a processor.

[0126] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for measuring three-dimensional topography, comprising: S11: projecting a plurality of randomly distributed speckle patterns onto the object to be measured by a rotating projection device; the rotating projection device comprises a light source, a speckle disk, a motor and a projection lens, wherein the motor drives the speckle disk to rotate, the light source irradiates a light beam to different positions on the speckle disk, and projects a plurality of speckle patterns onto the object to be measured through the projection lens; The rotary projection device further comprises: a collimating device, disposed between the light source and the speckle disk, configured to collimate the light emitted by the light source and then incident on the speckle disk; the speckle pattern is a speckle field of a preset local and global position randomly combined, so as to improve the anti-shadow and noise performance of the speckle pattern; S12: sequentially collecting multiple frames of deformation speckle patterns on the object under test; S13: determining a time-series variation sequence of the intensity of each spatial point on the measured object in the multiple frames of deformed speckle patterns; wherein the time-series variation sequence of the intensity comprises a sequence of grayscale values ​​of pixel points corresponding to each spatial point on the measured object in each frame of the deformed speckle pattern; S14: performing a correlation operation on the intensity time-series variation sequence of each spatial point and the intensity time-series variation distribution in the depth information calibration table, wherein the depth information calibration table is constructed according to the projection patterns of the multiple speckle patterns on multiple different depth planes; and S15: Determine a matching point with the highest correlation coefficient with each of the spatial points, and construct a three-dimensional shape of the object to be measured based on the depth of each matching point.

2. The measuring method according to claim 1, wherein the step S12 comprises: Multiple frames of deformed speckle patterns on the object under test are collected sequentially by only one camera.

3. The measuring method according to any one of claims 1 to 2, further comprising: The depth information calibration table is constructed in the following manner: S21: Projecting a randomly distributed speckle pattern onto a standard plane; S22: sequentially collecting multiple frames of deformed speckle patterns on the standard plane; S23: moving the standard plane by a first distance in the depth direction, and sequentially collecting multiple frames of deformed speckle patterns on the standard plane; S24: repeat step S23 until the standard plane is moved to a preset depth; S25: Based on the multi-frame deformed speckle patterns of multiple depth planes, a depth information calibration table of the intensity time-series change distribution of each point in space is established.

4. The measuring method according to any one of claims 1 to 2, wherein the step S12 further comprises: The time sequence correspondence or the corresponding frame start position between the acquired multi-frame deformed speckle pattern and the projected speckle pattern is determined.

5. The measuring method according to any one of claims 1 to 2, wherein the step S14 comprises: According to the grayscale value of the pixel point corresponding to each spatial point on the measured object in each frame of the deformed speckle pattern, the grayscale value of the pixel point changing in the time series in the multiple frames of the deformed speckle pattern is determined; the grayscale value of the pixel point corresponding to each spatial point and the grayscale value changing in the time series are correlated with the grayscale value of each pixel point having a corresponding relationship in the depth information calibration table and the grayscale value changing in the time series.

6. The measuring method according to any one of claims 1 to 2, wherein the step S15 comprises: The depth plane where the matching point having the highest temporal correlation coefficient with each spatial point is located is determined, and the three-dimensional shape of the measured object is constructed based on the depth information of each depth plane.

7. A method for establishing a depth information calibration table, comprising: S21: projecting a plurality of randomly distributed speckle patterns onto a standard plane through a rotating projection device; the rotating projection device comprises a light source, a speckle disk, a motor and a projection lens, wherein the motor drives the speckle disk to rotate, the light source irradiates a light beam to different positions on the speckle disk, and projects a plurality of speckle patterns onto the object to be measured through the projection lens; The rotary projection device further comprises: a collimating device, disposed between the light source and the speckle disk, configured to collimate the light emitted by the light source and then make it incident on the speckle disk; The speckle pattern is a speckle field that is a combination of preset local and global positions at random, so as to improve the anti-shadow and noise performance of the speckle pattern; S22: sequentially collecting multiple frames of deformed speckle patterns on the standard plane; S23: moving the standard plane by a first distance in the depth direction, and sequentially collecting multiple frames of deformed speckle patterns on the standard plane; S24: repeating step S23 until the standard plane is moved to a preset depth; and S25: Based on the multi-frame deformed speckle patterns of multiple depth planes, a depth information calibration table of the intensity time-series change distribution of each point in space is established.

8. The method according to claim 7, wherein the step S22 comprises: A plurality of frames of deformed speckle patterns on the standard plane are sequentially acquired by only one camera.

9. The method according to any one of claims 7-8, wherein the step S22 further comprises: The time sequence correspondence or the corresponding frame start position between the acquired multi-frame deformed speckle pattern and the projected speckle pattern is determined.

10. The method according to any one of claims 7-8, wherein the depth information calibration table comprises the grayscale values ​​of the pixels corresponding to each spatial point on the standard plane in each frame of the deformed speckle pattern on multiple depth planes, and the grayscale values ​​of the pixels changing in the time series in the multiple frames of the deformed speckle pattern.

11. A three-dimensional shape measurement system, comprising: a rotating projection device configured to project a randomly distributed speckle pattern; The randomly distributed speckle pattern is a preset local and global position randomly combined with a speckle field to improve the anti-shadow and noise performance of the speckle pattern; an image acquisition device configured to acquire a pattern; and A control device configured to control the rotation projection device and the image acquisition device to jointly perform the measurement method according to any one of claims 1 to 6 to construct a three-dimensional shape of the object to be measured; The rotary projection device comprises: light source; a speckle disk disposed downstream of the optical path of the light source and configured to modulate the light emitted by the light source into a randomly distributed speckle pattern; a motor configured to drive the speckle disk to rotate; a projection lens disposed downstream of the optical path of the speckle disk and configured to project the randomly distributed speckle pattern; and The collimating device is disposed between the light source and the speckle disk, and is configured to collimate the light emitted by the light source and then make it incident on the speckle disk.

12. The measurement system of claim 11, wherein the image acquisition device comprises only one camera.

13. The measurement system according to claim 11, further comprising: Standard plane; A mobile platform, used to carry the standard plane and configured to drive the standard plane to move in the depth direction; Wherein, the control device is further configured to control the rotating projection device, the image acquisition device and the mobile platform to jointly execute the method as described in any one of claims 7 to 10 to establish a depth information calibration table.

14. The measuring system according to any one of claims 11 to 13, wherein the speckle disk is provided with a grating encoder on the inner periphery and / or the outer periphery.

15. The measurement system according to claim 14, further comprising: a detection device configured to obtain a rotational position of the speckle disk based on the grating encoder; The control device is further configured to determine the time sequence correspondence or the corresponding frame start position between the acquired multiple frames of deformed speckle patterns and the projected speckle pattern according to the rotation position acquired by the detection device.

16. A computer-readable storage medium, comprising computer-executable instructions stored thereon, wherein the executable instructions, when executed by a processor, implement the method according to any one of claims 7 to 10 and the measurement method according to any one of claims 1 to 6.

Images