Temporal encoding method for space, three-dimensional reconstruction method and related products

Abstract

Embodiments of the present application provide a kind of time sequence encoding method of space, three-dimensional reconstruction method and related product.Time sequence encoding method includes: sequentially projecting N line encoding diagrams to the space to be measured;For each pixel position of the space to be measured, based on the pixel gray value of the same pixel position in the N first shooting image, the time sequence value corresponding to the gray scale change peak value in the gray scale change curve is determined as the folded time sequence value;The M pixel interval is encoded by the preset encoding rule;Based on the encoding rule and the encoding value corresponding to the pixel interval of each pixel position, the folded time sequence value corresponding to each pixel position is unfolded, and the unfolded time sequence value corresponding to each pixel position in the space to be measured is obtained.The embodiments of the present application simultaneously utilize space information and time domain information to time sequence encode each pixel position of the space to be measured, effectively solve the problem of data loss in high reflection or high dynamic area reconstruction.

Description

Technical Field

[0001] This invention relates to the field of three-dimensional imaging technology, and in particular to a temporal coding method for space, a three-dimensional reconstruction method and system, a calibration method for temporal and height mapping relationships, an electronic device, and a computer-readable storage medium. Background Technology

[0002] Machine vision is mainly used in industrial applications such as workpiece gripping, sorting, palletizing, quality inspection, assembly, and workpiece tracking. my country is a major manufacturing country, and many enterprises need to inspect, sort, and handle materials for their products, including various highly reflective metal parts. A key technology that needs to be studied in robotic gripping systems is how to accurately reconstruct the point cloud information of workpieces from scattered piles of metal parts using machine vision technology, and how to drive industrial robots to the correct posture to grasp the workpieces and place them in the appropriate processing positions through high-precision hand-eye calibration.

[0003] Current 3D reconstruction schemes based on fringe structured light analyze the fringe pattern projected onto the surface of the object under test, and use phase extraction technology to calculate the phase distribution of the fringe pattern. Since changes in the height of the object's surface cause spatial distortion of the fringes, the phase difference is calculated by comparing the distorted fringes with reference fringes. Combining this phase difference with the relationship between height change, the height information of the object can be calculated, thus achieving 3D reconstruction. However, in scenes with high reflectivity or high dynamic range, oversaturation in the highlight areas often leads to the loss of fringe information, affecting the accuracy of 3D reconstruction. Summary of the Invention

[0004] Based on the above situation, the main objective of this invention is to provide a temporal coding method for space, a three-dimensional reconstruction method, and a corresponding technical solution for related products. It utilizes spatial information and temporal information to perform temporal coding on the position of each pixel in the space under test, effectively solving the problem of data loss during reconstruction in high-reflectivity or high-dynamic areas.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0006] The first aspect of this invention provides a temporal encoding method for space in three-dimensional reconstruction, comprising: sequentially projecting N line-coded images onto the space to be measured, wherein, after each projection of a line-coded image, a first captured image is obtained, resulting in N first captured images and N temporal values ​​corresponding to the N first captured images; the line-coded image includes M reference lines uniformly arranged along the X-axis of the projection plane, wherein the M reference lines of the first line-coded image divide the space to be measured into M pixel intervals, and the M reference lines of the next line-coded image are formed by moving the M reference lines of the previous line-coded image along the X-axis of the projection plane by a step size of F pixels; in a line-coded image, the distance between any two adjacent reference lines is F*N-1 pixels;

[0007] For each pixel position in the space to be tested, based on the pixel grayscale values ​​of the same pixel position in the N first captured images, a grayscale change curve of grayscale value changing with time is fitted, and the time series value corresponding to the grayscale change peak in the grayscale change curve is determined as the folded time series value.

[0008] The M pixel intervals are encoded using a preset encoding rule to obtain M encoded values, and each pixel position within a pixel interval corresponds to the same encoded value.

[0009] Based on the encoding rules and the encoding values ​​corresponding to the pixel intervals where each pixel position is located, the folded timing values ​​corresponding to each pixel position are expanded to obtain the expanded timing values ​​corresponding to each pixel position in the test space.

[0010] Preferably, before sequentially projecting N line-coded images onto the space to be tested, the following steps are included:

[0011] The number of reference lines M in the N line-coded images is obtained based on the preset number of line-coded images N, the horizontal resolution E of the line-coded images, and the line shift step F of the line-coded images.

[0012] in, F is a positive integer less than or equal to 5.

[0013] Preferably, F = 1.

[0014] Preferably, encoding the M pixel intervals using a preset encoding rule includes:

[0015] The number of Gray-coded maps n is set according to the number M of the pixel intervals;

[0016] The period width of each gray code image is set according to the number N of the line code images, wherein the grayscale values ​​of all pixel positions within the same period width are equal;

[0017] Gray coding is performed on each pixel position in the test space using n Gray code images and their corresponding period widths to mark the M pixel intervals.

[0018] Preferably, the Gray code bit depth n needs to satisfy 2. n ≥M, and n takes the minimum integer value.

[0019] Preferably, the step of performing Gray coding on each pixel position in the test space using n Gray code images and their corresponding period widths includes:

[0020] n Gray code images are sequentially projected onto the space to be tested. Each projection of a Gray code image yields a second captured image, resulting in n second captured images. The Gray code values ​​formed by these n second captured images can identify at least M different pixel intervals. The period widths corresponding to the n Gray code images are, respectively, N, 2N, ..., 2N. n-1 N, or the period width, is: 2 n-1 NN,2 n-2 N, ... N.

[0021] Preferably, the step of performing Gray coding on each pixel position in the test space using n Gray code images and their corresponding period widths includes:

[0022] n+1+2 Gray code images are sequentially projected onto the space to be tested. Each projection of a Gray code image yields a second captured image, resulting in n+1+2 second captured images. The Gray code values ​​formed by these n+1+2 second captured images can identify at least M different pixel intervals. The period widths corresponding to the n+1 Gray code images are, respectively, 1 / 2N, N, 2N, ..., 2. n-1 N; or the period width is successively: 2 n-1 N, 2 n-2 N, ..., N, 1 / 2N; the other two Gray code diagrams are an all-black Gray code diagram and an all-white Gray code diagram.

[0023] Preferably, the step of performing Gray coding on each pixel position in the test space using n Gray code images further includes:

[0024] The binarization threshold of the Gray code image is determined based on the pixel grayscale values ​​of the second captured image corresponding to the all-black Gray code image and the all-white Gray code image.

[0025] For any pixel location in the second captured image, if the pixel grayscale value corresponding to the pixel location is greater than the binarization threshold, then the pixel grayscale value corresponding to the pixel location is set to 1; if the pixel grayscale value corresponding to the pixel location is less than or equal to the binarization threshold, then the pixel grayscale value corresponding to the pixel location is set to 0.

[0026] Preferably, for each pixel location in the test space, based on the change in pixel grayscale value at the same pixel location in the N first captured images, a grayscale change curve is fitted, and the time series value corresponding to the grayscale change peak in the grayscale change curve is determined as the folding time series value, including:

[0027] For any pixel location, obtain the N pixel grayscale values ​​corresponding to the pixel location in the N first captured images;

[0028] Fit the grayscale values ​​of the N pixels into a grayscale variation curve;

[0029] Obtain the time interval [t1, t2] in the grayscale change curve where the pixel grayscale value is greater than the grayscale threshold, and determine the time value corresponding to the grayscale change peak at the pixel position by combining the following formula;

[0030]

[0031] Where y is the temporal value corresponding to the peak of grayscale change, [t1,t2] is the temporal interval where the pixel grayscale value is greater than the grayscale threshold, and I t This represents the pixel grayscale value corresponding to time series t.

[0032] Preferably, the step of unfolding the folded timing value corresponding to each pixel position based on the Gray code values ​​corresponding to the M pixel intervals includes:

[0033] For any pixel position, obtain the folding timing value and Gray code value corresponding to the pixel position, and determine the unfolding timing value corresponding to the pixel position by combining the following formula: T(i,j)=kN+t(i,j);

[0034] Where k is the Gray code value corresponding to the pixel interval where pixel position (i,j) is located, t(i,j) is the folding time sequence value corresponding to pixel position (i,j), and T(i,j) is the unfolding time sequence value corresponding to pixel position (i,j).

[0035] A second aspect of this invention provides a method for calibrating the temporal-height mapping relationship in 3D reconstruction, comprising:

[0036] Step S100: Place a calibration object with known height data in the calibration plane of the calibration space to be calibrated. The height data is the height value of each pixel position in the calibration space relative to the calibration plane. The calibration space to be calibrated has a preset temporal and height mapping relationship to be determined. The relationship includes multiple unknown calibration polynomial coefficients.

[0037] Step S200: Using any of the timing coding methods in the first aspect described above, obtain the unfolded timing value corresponding to each pixel position in the calibration space;

[0038] Step S300: Determine whether the current number of calibration measurements P is met. If not, control the calibration plane to rise a set distance and continue to execute step S200; if yes, execute step S400.

[0039] Step S400: Substitute the altitude data and deployment time series values ​​corresponding to the calibration measurement of group P into the time series-altitude mapping formula to determine the unknown calibration polynomial coefficients, thereby determining the time series-altitude mapping formula.

[0040] Preferably, the relation includes:

[0041] h(i,j)=p w T(i,j) w +p w-1 T(i,j) w-1 +…+p1T(i,j)+p0

[0042] Where h(i,j) is the height data corresponding to pixel position (i,j), from p0 to p w To calibrate the polynomial coefficients, w is the highest order of the polynomial, and T(i,j) is the expanded temporal value corresponding to pixel position (i,j).

[0043] A third aspect of the present invention provides a three-dimensional reconstruction method, comprising:

[0044] Using any of the timing coding methods in the first aspect above, obtain the unfolded timing value corresponding to each pixel position in the space to be tested;

[0045] Substitute the unfolded timing value into the timing-height mapping formula to obtain the height data corresponding to each pixel position in the test space;

[0046] Based on the height data corresponding to each pixel position, the space to be tested is reconstructed in three dimensions.

[0047] Preferably, the time-series-height mapping relationship is as follows:

[0048] h(i,j)=p w T(i,j) w +p w-1 T(i,j) w-1 +…+p1T(i,j)+p0

[0049] Where h(i,j) is the height data corresponding to pixel position (i,j), from p0 to p wThe coefficients are the preset calibration polynomials, w is the highest order of the polynomial, and T(i,j) is the expansion time value corresponding to pixel position (i,j).

[0050] The fourth aspect of the present invention provides a timing coding apparatus for executing any of the timing coding methods of the first aspect described above.

[0051] A third-dimensional reconstruction system is provided in the fifth aspect of the present invention, the system comprising:

[0052] Projection device for projecting line-coded diagrams;

[0053] Imaging device for capturing a first image corresponding to a line-coded diagram;

[0054] A control device for executing the three-dimensional reconstruction method described in the third aspect above.

[0055] The sixth aspect of the present invention provides a computer program stored thereon, which, when executed by a processor, implements the three-dimensional reconstruction method described in the third aspect above.

[0056] An electronic device provided in the seventh aspect of the present invention includes a storage medium storing a computer program, which, when executed by a processor, implements the three-dimensional reconstruction method described in the third aspect above.

[0057] In 3D reconstruction schemes based on striped structured light, it is necessary to calculate the relevant height information by identifying the grayscale values ​​of the pixels corresponding to the stripes. If only spatial information (i.e., the grayscale value corresponding to a specific pixel position) is collected during measurement, when encountering scenes with high reflectivity or high dynamic range, when the stripes pass through a highlight area, the pixels corresponding to the stripes and their neighboring pixels are in a highlight saturation state (i.e., the grayscale value of the pixel corresponding to the bright line and the grayscale values ​​of other pixels have reached or are close to the grayscale extreme value), making it impossible to effectively identify the accurate position of the pixels corresponding to the stripes, resulting in the loss of data corresponding to these pixels.

[0058] The present invention provides a solution through the above-mentioned aspects. By performing line encoding on the test space, the peak grayscale change of each pixel position in the test space is associated with the corresponding temporal value. The test space is divided into M pixel intervals to obtain folded temporal values ​​for the M pixel intervals. Then, the folded temporal values ​​are unfolded using Gray encoding to obtain the unfolded temporal value corresponding to each pixel position in the test space (i.e., temporal encoding is completed for each pixel position in the test space). Since the peak grayscale change in the embodiments of the present invention is the peak of the grayscale change curve, and the grayscale change curve is obtained by fitting the grayscale value changes of multiple images taken at the same pixel position, regardless of the influence of the highlight area on the actual grayscale value of each pixel position, a peak grayscale change can be obtained at the corresponding pixel position through simulation fitting. This peak grayscale change is then converted into a temporal value through the grayscale change curve, so an unfolded temporal value can be obtained for each pixel position. In the subsequent 3D reconstruction process, if the 3D reconstruction is performed based on the unfolded temporal value of each pixel position, and then the unfolded temporal value is converted back into height data according to the temporal-height mapping relationship, since the temporal sequence is not affected by the highlight area, the influence of the highlight area on the acquisition of height data can be avoided. This effectively solves the problem of data loss when traditional line structured light is reconstructed in high-reflection or high-dynamic-range areas, and has significant advantages for the measurement of high-reflection or high-dynamic-range scenes.

[0059] Other beneficial effects of the present invention will be explained in detail through the introduction of specific technical features and technical solutions in specific embodiments. Those skilled in the art should be able to understand the beneficial technical effects brought about by these technical features and technical solutions through the introduction of these technical features and technical solutions. Attached Figure Description

[0060] Preferred embodiments of the present invention will now be described with reference to the accompanying drawings. In the drawings:

[0061] Figure 1 This diagram illustrates a flowchart of a temporal coding method for space in three-dimensional reconstruction according to an embodiment of the present invention.

[0062] Figure 2 This diagram illustrates a line coding diagram in an embodiment of the present invention.

[0063] Figure 3 This is a schematic diagram of a 40-step line coding diagram in an embodiment of the present invention;

[0064] Figure 4 This diagram illustrates a flowchart of a method for determining folding timing values ​​in an embodiment of the present invention.

[0065] Figure 5 This diagram illustrates a timeline shift center extraction embodiment of the present invention.

[0066] Figure 6 This diagram illustrates a grayscale variation curve in an embodiment of the present invention.

[0067] Figure 7 This diagram illustrates an incomplete grayscale variation curve in an embodiment of the present invention.

[0068] Figure 8 This diagram illustrates a grayscale change curve after filling in an embodiment of the present invention.

[0069] Figure 9 This shows a schematic diagram of a 5-bit Gray code map + complementary Gray code map in an embodiment of the present invention;

[0070] Figure 10 This diagram illustrates the period width corresponding to the 5-bit Gray code map + complementary Gray code map in an embodiment of the present invention.

[0071] Figure 11 This diagram illustrates a folding timing sequence in an embodiment of the present invention.

[0072] Figure 12 This diagram illustrates the unfolding timing in an embodiment of the present invention.

[0073] Figure 13 This diagram illustrates the folding timing of the 100th row of pixels in an embodiment of the present invention.

[0074] Figure 14 This diagram illustrates the timing sequence of the expansion of the 100th row of pixels in an embodiment of the present invention.

[0075] Figure 15 A flowchart illustrating a calibration method for the timing-height mapping relationship in an embodiment of the present invention is shown.

[0076] Figure 16 This diagram illustrates a flowchart of a three-dimensional reconstruction method according to an embodiment of the present invention.

[0077] Figure 17 A schematic diagram of an electronic device for implementing a three-dimensional reconstruction method is shown in an embodiment of the present invention. Detailed Implementation

[0078] The present invention is described below based on embodiments, but the present invention is not limited to these embodiments. In the following detailed description of the present invention, some specific details are described in detail, but well-known methods, processes, procedures, and elements are not described in detail in order to avoid obscuring the essence of the present invention.

[0079] Furthermore, those skilled in the art should understand that the accompanying drawings provided herein are for illustrative purposes only and are not necessarily drawn to scale.

[0080] Unless the context explicitly requires it, the words "comprising," "including," and similar terms throughout the specification and claims should be interpreted as encompassing rather than being exclusive or exhaustive; that is, meaning "including but not limited to."

[0081] In the description of this invention, it should be understood that the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance. Furthermore, in the description of this invention, unless otherwise stated, "a plurality of" means two or more.

[0082] The specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0083] In the scheme of 3D reconstruction using structured light, a structured light with a set pattern is projected onto the surface of an object in the space to be measured by a projection device. An imaging device that works with the projection device captures the structured light reflected back from the surface of the object in the space to be measured. Since the structured light is modulated on the surface of the object, the reflected structured light carries the surface information of the object. By demodulating the reflected structured light, the 3D morphological information of the object can be obtained, and then the 3D morphology of the object surface can be reconstructed.

[0084] As mentioned earlier, traditional 3D reconstruction schemes based on striped structured light only collect spatial information of the image (i.e., the position information of the center of the reference line in the line structure image). In high-reflection or high-dynamic-range scenes, the reflected image may contain a high-exposure area where the gray values ​​of the pixels in this area reach or approach the gray-scale extreme values, making it impossible to effectively distinguish the edge information of the object, resulting in data loss during 3D reconstruction.

[0085] In view of this, this invention proposes a temporal coding method for space in 3D reconstruction. It uses both spatial and temporal information to perform temporal coding on the position of each pixel in the space under test, which can avoid the influence of specular areas on the acquisition of height data and effectively solve the problem of data loss during reconstruction in high-reflectivity or high-dynamic areas.

[0086] Please see Figure 1 , Figure 1 This diagram illustrates a flowchart of a temporal encoding method for space in 3D reconstruction according to an embodiment of the present invention. The present invention provides a temporal encoding method for space in 3D reconstruction, comprising steps S10 to S40.

[0087] In step S10, N line-coded images are projected sequentially onto the space to be tested. Each projection of a line-coded image yields a first captured image, resulting in N first captured images. Furthermore, for each captured first image, a corresponding time value is recorded (optionally, the time value for each captured first image can be calculated after all N first captured images have been captured, or the capturing time sequence can be recorded in real time; the specific method depends on actual needs and is not limited here). Therefore, N first captured images correspond to N recorded time values ​​(t = 1, 2, ..., N-1, N). It should be noted that in this embodiment of the invention, for ease of explanation and example, numerical sequences are used as examples for the time values. In practical applications, the time values ​​can correspond to actual time values, such as t1 = 8:01:01, t2 = 8:01:02, ..., tN = 8:01:0N. The specific time interval between the N time values ​​depends on the capturing frequency of the imaging device and is not limited here.

[0088] Figure 2 This diagram illustrates a line encoding pattern according to an embodiment of the present invention. Figure 2 As shown, the line-coded diagram includes M reference lines evenly arranged along the X-axis of the projection plane (i.e., Figure 2 (The bright lines in the image). Each reference line has a width of one pixel. Specifically, in a line-coded image, the distance between any two adjacent reference lines is F*N-1 pixels. For N consecutive line-coded images, the reference lines of any two adjacent line-coded images are staggered by a width of F pixels on the X-axis of the projection plane. Here, F represents the line shift step, which is a positive integer less than or equal to 5. A set of N consecutive line-coded images, under continuous projection, makes the reference lines appear to move along the X-axis; therefore, a set of consecutive line-coded images is also called a set of line-shifted images. Here, F is the line shift step (i.e., the distance the reference line moves along the X-axis each time), and N is the number of line shift steps. For example, the first reference line of the first line-coded image is... Figure 2 The bright line corresponding to the first pixel on the X-axis, if the projection resolution of the line-coded image is set to E, then (That is, the number of reference lines is equal to the horizontal resolution divided by (line shift step * number of line-coded maps) and then rounded down). Therefore, the M reference lines of the first line-coded map divide the space to be tested into M pixel regions of width F * N. It should be noted that in this embodiment of the invention, N is not equal to E, and the minimum value of M is 2, meaning that the space to be tested is divided into at least two pixel intervals.

[0089] For example, assume the projection resolution of the line-coded image is 1280*720 pixels, the number of line-coded images is N=40, and the line shift step F is 1 pixel. Figure 3 This illustrates a schematic diagram of a 40-step line coding diagram in an embodiment of the present invention, as shown below. Figure 3As shown, the spacing between any two adjacent reference lines in each line-coded image is 39 pixels. The pattern of the first line-coded image can be referenced. Figure 2 In the image, each bright line (reference line) occupies 1 pixel in the X-axis direction. The first bright line in the first line-coded image is located in the first column, and the pixel value of this column is set to 1 (0 represents black and 1 represents white in a binary image). Then, the next bright line is generated periodically at line intervals until there are 32 (1280 ÷ 40 = 32) bright lines in the image, at which point the first line-coded image is generated. Subsequent line-coded images are generated by moving each bright line one pixel to the right (F = 1); this process continues until 40 line-coded images are obtained. The j-th line-coded image can be represented by the following formula:

[0090]

[0091] Where j = 1, 2, ..., N, u and v are the horizontal and vertical coordinates of the image, N is the number of line-coded images, and % represents the modulo operation.

[0092] In step S20, for each pixel position in the space to be tested, based on the pixel gray values ​​of the same pixel position in N first captured images, a gray value change curve is fitted to show the change of gray value over time, and the time value corresponding to the gray value change peak in the gray value change curve is determined as the folded time value.

[0093] In one embodiment of step S20, steps S21 to S25 may be included; please refer to [link / reference]. Figure 4 , Figure 4 This diagram illustrates a flowchart of a method for determining folded timing values ​​according to an embodiment of the present invention.

[0094] In step S21, a pixel position is selected, and the N pixel grayscale values ​​corresponding to that pixel position in the N first captured images are obtained.

[0095] In step S22, the gray values ​​of N pixels are fitted into a gray-scale change curve.

[0096] In step S23, the time interval [t1, t2] where the pixel gray value in the gray-scale change curve is greater than the gray-scale threshold is obtained, and the time value corresponding to the gray-scale change peak at the current pixel position is determined by combining the following formula of gray-scale centroid method.

[0097]

[0098] Where y is the temporal value corresponding to the peak of grayscale change, [t1,t2] is the temporal interval where the pixel grayscale value is greater than the grayscale threshold, and I t This represents the pixel grayscale value corresponding to time series t. Figure 5 This diagram illustrates the timing line shift center extraction in an embodiment of the present invention, as shown below. Figure 5 As shown in the figure, I is marked th This is the grayscale threshold.

[0099] The grayscale threshold mentioned above is a lower limit set for valid data. Only data with pixel grayscale values ​​greater than the grayscale threshold are selected for subsequent algorithm processing.

[0100] In step S24, it is determined whether there are any unselected pixel positions in the current test space. If there are, the process returns to step S21; otherwise, step S25 is executed.

[0101] In step S25, the folding timing value corresponding to each pixel position in the test space is output.

[0102] For example, assume the projection resolution of the line-coded image is 1280*720 pixels, the number of line-coded images is 40, and the line shift step is 1 pixel. The grayscale value curve distribution of a pixel position over 40 time sequences (t = 1, 2, ..., 39 and 40) is as follows: Figure 6 As shown, Figure 6 This diagram illustrates a grayscale variation curve in an embodiment of the present invention. Figure 6 If the gray value of a pixel does not clearly reach its peak at a certain time t, or does not perfectly conform to the Gaussian curve distribution (i.e., at this time, it may be because the reference line passes through a highlight area, making it impossible to effectively identify the gray value of the pixel corresponding to the reference line), then the time-series centroid value can be weighted and calculated using the formula of the gray-scale centroid method described above. This simulation can determine at a certain time t (e.g., t = 16.25) when the gray value of the pixel reaches its peak (i.e., at what time t does the corresponding pixel gray value reach its maximum).

[0103] In other scenarios of this invention, due to the positional differences of each pixel, the grayscale value distribution of some pixels does not perfectly conform to a Gaussian distribution, and the grayscale change curve is as follows: Figure 7 As shown, Figure 7 This diagram illustrates an incomplete grayscale change curve in an embodiment of the present invention. In this embodiment, the calculation is performed at the moment when the grayscale value of a certain pixel reaches its maximum. Since the grayscale change curve in this embodiment is obtained by fitting N discrete pixel grayscale values, a grayscale change peak may not necessarily appear among the N pixel grayscale values. If a grayscale change curve that conforms to or approximates a Gaussian distribution cannot be fitted, the time series value corresponding to the obtained grayscale change peak may not be accurate enough.

[0104] For pixels whose grayscale value distribution does not conform to the Gaussian distribution standard, the latter part and the former part of the current grayscale change curve can be extracted. Figure 7The image (defined by the dashed line) is simulated, and the data is filled into the front and rear regions of the original grayscale curve, respectively. Figure 8 The simulated area (defined by the dashed line) yielded the following results: Figure 8 The grayscale variation curve shown is Figure 8 This diagram illustrates a grayscale change curve after filling in the gaps in an embodiment of the present invention. The time-series value corresponding to the peak grayscale change is then obtained using the filled grayscale change curve. Figures 6 to 8 In the diagram, the horizontal axis corresponds to the time series value, and the vertical axis corresponds to the grayscale value. It should be noted that in practical applications, if encountering... Figure 7 In this case, the discrete data corresponding to the time interval [t1,t2] can also be substituted into the formula of the gray-scale centroid method mentioned above for calculation, but because... Figure 7 The data continuity is poor, so the results are not accurate enough. Preferably, the above method should be used first... Figure 7 The data is truncated and fitted to obtain... Figure 8 Substitute Figure 8 By calculating the continuous data, we can obtain more accurate time-series values ​​corresponding to the peak values ​​of grayscale changes.

[0105] In the above scheme, the peak value of the gray change is determined to be the position of the reference line in the line coding image (the gray value of the bright line is the largest). During the line shift of the N line coding images, the temporal values ​​of the corresponding N first captured images are arranged in a near Gaussian distribution. Therefore, the weighted center position of the gray value is extracted according to the gray centroid method, which is the temporal value corresponding to the pixel position.

[0106] Regarding the concept of folded timing values, as mentioned above, each time a line-coded image is projected, a first captured image is taken, and a corresponding timing value is recorded. Assuming the line shift step is 1 pixel (i.e., F = 1), after taking N consecutive images, if the first reference line of each line-coded image is used as the observation viewpoint for the line shift, the first reference line is equivalent to traversing the first pixel region of the test space, and N time-series values ​​are recorded during the traversal. If the second reference line of each line-coded image is used as the observation viewpoint for the line shift, the second reference line is equivalent to traversing the second pixel region of the test space, and N time-series values ​​are also recorded during the traversal. The subsequent Mth reference line follows the same pattern. Note that the N time-series values ​​recorded during the traversal of each reference line are the same (e.g., the time-series values ​​t = 1, 2, ..., N-1 and N when the first reference line is traversed; and the time-series values ​​when the second reference line is traversed are also t = 1, 2, ..., N-1 and N). In terms of timing, this is equivalent to folding M pixel intervals into one pixel interval. Therefore, the time-series value corresponding to the grayscale change peak is the folded time-series value.

[0107] In other scenarios of this invention, the step size of the line shift can be set according to actual computational needs. For example, assuming the step size of the line shift is 2 pixels (i.e., F = 2), it is equivalent to performing a half-sampling on the original process of traversing all pixels. Although some accuracy is sacrificed, the sampled grayscale data can still be fitted into a grayscale change curve, and the final timing coding result will not have a large deviation, while reducing the data processing overhead and improving the efficiency of timing coding.

[0108] In step S30, the M pixel intervals are encoded according to a preset encoding rule to obtain M encoded values, and each pixel position within a pixel interval corresponds to the same encoded value.

[0109] As mentioned earlier, since the test space is divided into M pixel intervals, and these M pixel intervals are temporally folded into a single pixel interval, the folded temporal values ​​corresponding to each pixel interval need to be unfolded sequentially to obtain the accurate temporal value corresponding to each pixel position in the test space. Therefore, before performing the unfolding operation, these M pixel intervals need to be encoded sequentially to obtain the encoded values ​​corresponding to each of the M pixel intervals. During temporal unfolding, the M encoded values ​​are decoded sequentially according to a preset encoding rule, so that the decoding results can be combined with the corresponding folded temporal values ​​to calculate the corresponding unfolded temporal value. It should be noted that any encoding rule that can sequentially encode pixel intervals (and can reverse the encoding sequence according to the original encoding rule) can be used as the preset encoding rule in this embodiment of the invention.

[0110] In step S40, based on the encoding rules and the encoding value corresponding to the pixel interval where each pixel position is located, the folded timing value corresponding to each pixel position is unfolded to obtain the unfolded timing value corresponding to each pixel position in the test space.

[0111] For example, please refer to Figure 11 , Figure 12 , Figure 13 and Figure 14 . Figure 11 This diagram illustrates a folding timing sequence in an embodiment of the present invention. Figure 13 This diagram illustrates the folding timing of the 100th row of pixels in an embodiment of the present invention. Figure 12 This diagram illustrates the unfolding timing in an embodiment of the present invention. Figure 14 This diagram illustrates the unfolding timing of the 100th row pixel position in an embodiment of the present invention. Assuming the projection resolution of the line-coded image is 1280*720 pixels, the resolution of the captured image is 2048*1376 pixels, the number of line-coded images is 40, and the line shift step is 1 pixel, the folding timing image before level-wise unfolding is as follows: Figure 11As shown, the unfolded timing diagram after timing expansion is as follows: Figure 13 As shown. The folding timing value corresponding to the horizontal pixel at the 100th row of the captured image is as follows: Figure 12 As shown in the figure (the vertical axis represents the timing value, and the horizontal axis represents the pixel), correspondingly, after hierarchical expansion, the expanded timing value corresponding to the horizontal pixel at the 100th row position is as follows: Figure 14 As shown.

[0112] In step S30 above, Gray code can be used as the preset encoding rule. In structured light-based 3D reconstruction, Gray code is a type of encoding used to assist phase unrolling. Since only one bit differs between any two adjacent Gray code values ​​(i.e., a set of Gray code values ​​can generate at most one bit error), it has strong anti-interference capabilities, is simple to encode, and has a wide range of applications.

[0113] Specifically, each pixel position in the test space can be Gray-coded using n Gray code images, and the n Gray code images mark the test space into M pixel intervals. For example, before Gray coding, the number of Gray code bits n required can be determined based on the number of line codes N and the set horizontal resolution E. The set horizontal resolution refers to the preset horizontal resolution of the line code image or Gray code image. The Gray code bit number n needs to satisfy 2... n The condition is ≥M, and n takes the minimum integer value.

[0114] For example, in one embodiment of step S30, n Gray code images can be projected sequentially onto the space to be tested. Each projection of a Gray code image yields a second captured image, resulting in n second captured images. The Gray code values ​​formed by the n second captured images can identify at least M different pixel intervals. The period widths corresponding to the n Gray code images are respectively: N, 2N, ..., 2 n-1 N, or the period width, is: 2 n-1 N, 2 n-2 N, ... N.

[0115] For example, in this embodiment of the invention, all pixel positions within the test space are Gray-coded. Please refer to... Figure 9 , Figure 9 This diagram illustrates a 5-bit Gray code image + complementary Gray code image in an embodiment of the present invention. In the diagram, GC1, GC2, GC3, GC4, and GC5 represent the first to fifth Gray code images, respectively, and GC6 is the complementary Gray code image. The five Gray code images GC1, GC2, GC3, GC4, and GC5 can divide the horizontal pixels into 32 pixel intervals (n is the number of Gray code bits, i.e., 2^n x 1 ... nEach Gray code chart represents a specific bit in the Gray code (within a range of intervals), where black represents 0 and white represents 1.

[0116] In this embodiment of the invention, the period width represents the length of a Gray code image where the Gray code values ​​remain unchanged (i.e., the grayscale values ​​of all pixels within the same period width are equal). The grayscale values ​​can be identified using "0" and "1" respectively. In practical applications, a grayscale value of "0" does not necessarily mean the grayscale value is 0; it may indicate that the grayscale value of the pixel is below the binarization threshold. Similarly, a grayscale value of "1" may indicate that the grayscale value of the pixel is above the binarization threshold. It should be noted that the method of Gray coding by projecting a Gray code image is only one example. It is understood that in practical applications, other methods can be used to implement Gray coding (e.g., simulating the image captured by the Gray code image in the test space through layer overlay in a computer). No specific limitations are made here.

[0117] In another embodiment of step S30, since edge transitions may occur during the hierarchical expansion of Gray code, complementary Gray codes can be introduced during Gray coding to improve the stability and accuracy of decoding. Specifically, n+1+2 Gray code images can be projected sequentially onto the space to be tested, wherein each projection of a Gray code image yields a second captured image, resulting in n+1+2 second captured images; the Gray code values ​​formed by the n+1+2 second captured images can identify at least M different pixel intervals; wherein the period widths corresponding to the n+1 Gray code images are respectively: 1 / 2N, N, 2N, ..., 2 n-1 N; or the period width is successively: 2 n-1 N, 2 n-2 N, ..., N, 1 / 2N; the other two Gray code diagrams are an all-black Gray code diagram and an all-white Gray code diagram.

[0118] In another embodiment of step S30, the Gray code image, besides being used to set the coding bits of corresponding pixels, can also be used to assist in marking the pixel grayscale values ​​corresponding to the second captured image. Specifically, n+2 Gray code images can be projected sequentially onto the space to be tested, wherein each projection of a Gray code image yields a second captured image, resulting in n+2 second captured images; the Gray code values ​​formed by the n+2 second captured images can identify at least M different pixel intervals. The period widths corresponding to the n Gray code images are respectively: N, 2N, ..., 2 n-1 N, or the period width, is: 2 n-1 N, 2 n-2 N, ... N; the other two Gray code diagrams are an all-black Gray code diagram and an all-white Gray code diagram.

[0119] For example, the binarization threshold of the second captured image can be determined based on the pixel grayscale values ​​of the second captured image corresponding to the all-black Gray code image and the all-white Gray code image. For instance, the second captured image corresponding to the all-black Gray code image and the all-white Gray code image is acquired by an imaging device, and the binarization threshold I is calculated based on the following formula. threshold :

[0120]

[0121] Among them, I w I represents the pixel grayscale value of the second captured image corresponding to the all-white Gray code image. b This represents the pixel grayscale value of the second captured image corresponding to the all-black Gray code image. For any pixel location in the second captured image, if the pixel grayscale value corresponding to the pixel location is greater than the binarization threshold, then the pixel grayscale value corresponding to the pixel location is set to 1; if the pixel grayscale value corresponding to the pixel location is less than or equal to the binarization threshold, then the pixel grayscale value corresponding to the pixel location is set to 0.

[0122] For example, suppose the period widths corresponding to the above n+1 Gray code images are respectively: 2 n-1 N, 2 n-2 For N, ..., N, 1 / 2N, the Gray code diagram corresponding to a period width of 1 / 2N is the complementary Gray code diagram. Please refer to... Figure 10 , Figure 10 This diagram illustrates the period widths corresponding to the "5+1" Gray code diagrams in an embodiment of the present invention. Wherein, k1 represents the encoded value (decimal) corresponding to the Gray code diagram, and k2 represents the encoded value (decimal) corresponding to the complementary Gray code diagram. Figure 10 In this context, GC6 represents the period width corresponding to the complementary Gray code image. Assuming the projected resolution of the line-coded image is 1280*720 pixels, the number of line-coded images is 40, and the line shift step is 1 pixel, then Gray coding needs to identify 32 different pixel intervals, n=5. In addition, a complementary Gray code image is added. The period widths corresponding to these "5+1" Gray code images are respectively: 640 (pixels), 320, 160, 80, 40, and 20. When calculating the unfolded timing value based on the Gray code, the following complementary Gray code formula can be used.

[0123]

[0124] Where t(i,j) represents the folded timing value, T(i,j) represents the unfolded timing value, k1 represents the encoded value (decimal) corresponding to the Gray code diagram, and k2 represents the encoded value (decimal) corresponding to the complementary Gray code diagram. From the above formula, it can be seen that when the folded timing value is within the interval t(i,j)≤-(F*N) / 4 or t(i,j)≥(F*N) / 4, this interval is prone to edge transitions. In this case, using complementary Gray code for decoding, as long as the Gray code offset does not exceed (F*N) / 4, the accurate timing value can be restored using complementary Gray code.

[0125] Specifically, in another embodiment of step S40, for any pixel position, the folding timing value and Gray code value corresponding to the pixel position are obtained, and the unfolding timing value corresponding to the pixel position is determined by combining the following formula.

[0126] T(i,j)=kN+t(i,j)

[0127] Where k is the Gray code value corresponding to the pixel interval where pixel position (i,j) is located, t(i,j) is the folding timing value corresponding to pixel position (i,j), and T(i,j) is the unfolding timing value corresponding to pixel position (i,j). In practical applications, edge jumps may occur when performing level-wise unfolding of Gray codes. Therefore, this embodiment of the invention sets GC6 as the complementary Gray code map. When the decoding region is located at the edge of the pixel region, the complementary Gray code value k2 is used for unfolding; when the decoding region is located at a non-edge position of the pixel region, the normal Gray code value k1 is used for unfolding. Specifically, please refer to the above complementary Gray code formula. If the folding timing value is in the interval t(i,j)≤-(F*N) / 4 or t(i,j)≥(F*N) / 4, then the unfolded T(i,j)=t(i,j)+k2*F*N, or T(i,j)=t(i,j)+(k2-1)*F*N. If the folding timing value is in Within the interval, the expanded T(i,j) = t(i,j) + k1*F*N.

[0128] In the temporal coding method of the three-dimensional reconstruction space in the embodiments of the present invention, the gray-scale change peak of each pixel position in the test space is associated with the corresponding temporal value by performing line coding on the test space, and the test space is divided into M pixel intervals to obtain the folded temporal values ​​of the M pixel intervals. Then, the folded temporal values ​​are unfolded by Gray coding to obtain the unfolded temporal values ​​corresponding to each pixel position in the test space (that is, the temporal coding of each pixel position in the test space is completed).

[0129] First, since the grayscale change peak in this embodiment is the peak of the grayscale change curve, which is obtained by fitting the grayscale value changes of multiple captured images at the same pixel location, regardless of the influence of the highlight area on the actual grayscale value of each pixel location, a grayscale change peak can be obtained at the corresponding pixel location through simulation fitting. This grayscale change peak is then converted into a temporal value using the grayscale change curve, resulting in an unfolded temporal value for each pixel location. In the subsequent 3D reconstruction process, if 3D reconstruction is performed based on the unfolded temporal value of each pixel location, and the unfolded temporal value is then converted back to height data according to the temporal-height mapping formula, the temporal data is not affected by the highlight area. Therefore, the influence of the highlight area on the acquisition of height data can be avoided, effectively solving the problem of data loss during reconstruction in high-reflection or high-dynamic-range areas using traditional line structured light. This provides a significant advantage for measuring high-reflection or high-dynamic-range scenes.

[0130] Secondly, this embodiment of the invention uses a line-coded pattern including multiple reference lines to line-code the space under test, dividing the space under test into multiple pixel intervals. The grayscale change peak of each pixel position in each pixel interval is associated with the corresponding time-series value, so that multiple pixel intervals are folded into one pixel interval in time. Subsequently, the pixel intervals can be encoded and then unfolded in sequence. This reduces the amount of line-coded pattern required by folding pixel intervals, thereby greatly reducing the requirements for hardware performance and greatly improving processing efficiency and product performance.

[0131] Furthermore, in the embodiments of the present invention, both line coding and Gray coding can be encoded quickly through projection, which is more efficient in processing data compared to structured light formed by traditional laser scanning.

[0132] In the 3D reconstruction scheme using line structured light in this invention embodiment, since light is modulated differently by surfaces at different heights, the temporal value corresponding to the peak grayscale change at any given pixel location will differ across N consecutive first-capture images at different heights. For example, all other things being equal, for any pixel location, the higher the height data of that pixel location, the earlier the peak grayscale change will be reached on the corresponding temporal axis. Therefore, a correlation between height change and temporal change can be established through prior calibration, allowing the height data of each pixel location to be calculated using the calibrated temporal-height mapping formula after temporal encoding of the measured space.

[0133] In practical applications, the modulation of light by different object surfaces is a complex process involving various phenomena such as light reflection, refraction, and scattering. These phenomena work together to modulate light as it passes through different object surfaces, resulting in changes in light intensity, phase, polarization state, and reflection time. Therefore, before measuring the height data of an object, multiple sets of calibration objects with known different height data (the calibration objects and the object to be measured can be the same or similar materials) can be used. Under the same environment (e.g., lighting and temperature / humidity conditions), the calibration space is subjected to the same line encoding (i.e., the N, M, and F parameters are consistent) to eliminate interference from factors other than height data. This ensures that the temporal values ​​corresponding to the grayscale change peaks at each pixel location within the calibration space can reflect the changes in height data, forming an accurate temporal-height mapping relationship. Specifically, this embodiment of the invention also provides a calibration method for the temporal-height mapping relationship in 3D reconstruction, including steps S100 to S400. Please refer to... Figure 15 , Figure 15 A flowchart illustrating the calibration method for the timing-height mapping relationship in an embodiment of the present invention is shown.

[0134] In step S100, a calibration object with known height data is placed within the calibration plane of the calibration space to be calibrated. In this embodiment of the invention, the space to be measured refers to the space where the object to be measured is placed, and the calibration space refers to the space calibrated using the calibration object. The height data refers to the height values ​​of each pixel position within the calibration space relative to the calibration plane. Furthermore, for the calibration space, a preset temporal-height mapping relationship is established, which includes multiple unknown calibration polynomial coefficients.

[0135] Specifically, the calibration plane can be controlled by a displacement mechanism to make precise displacements in the height direction. After each calibration measurement of the current height is completed, it moves a set distance in the same direction and continues to the next set of calibration measurements. For example, in order to obtain an accurate temporal and height mapping relationship of the current space, 10 calibration measurements can be performed, with each measurement moving 1 mm.

[0136] In step S200, any of the timing coding methods described in the above embodiments are used to obtain the unfolded timing value corresponding to each pixel position in the calibration space.

[0137] In step S300, it is determined whether the preset number of calibration measurements P is met. If not, the calibration plane is controlled to rise by a set distance, and step S200 is continued; if yes, step S400 is executed.

[0138] In step S400, the height data and unfolded timing values ​​corresponding to the P-group calibration measurements are substituted into the timing-height mapping formula to determine the corresponding calibration polynomial coefficients. Here, the height data in step S400 refers to the height value of each pixel position in the calibration space relative to the calibration plane during the calibration measurement. The height data can be calculated based on the height information of the calibration object and the current height value of the calibration plane.

[0139] Specifically, the above time-series-height mapping relationship can be expressed as:

[0140] h(i,j)=p w T(i,j) w +p w-1 T(i,j) w-1 +…+p1T(i,j)+p0

[0141] In step S400, h(i,j) is the known height data of pixel position (i,j), p0 to p w Here, w represents the coefficients of the unknown calibration polynomial, w is the highest order of the polynomial, and T(i,j) is the unfolded temporal value corresponding to pixel position (i,j). In this embodiment of the invention, the height data and unfolded temporal values ​​obtained from the P groups of calibration measurements, respectively, can be substituted into the temporal-height mapping formula to obtain o0 to p. w The specific values ​​of terms w+1 are then used to obtain the known values ​​from p0 to p. w The temporal and height mapping relationship.

[0142] It is understood that the method for temporal coding of the calibration space in this application embodiment can adopt any of the temporal coding methods in the above embodiments, which will not be repeated here.

[0143] In this embodiment of the invention, after the temporal and height mapping relationship of the space under test is calibrated, the height data corresponding to each pixel position in the space under test can be obtained by temporal encoding. Specifically, a three-dimensional reconstruction method in this embodiment of the invention includes steps S1 to S3. Please refer to... Figure 16 , Figure 16 A flowchart of a three-dimensional reconstruction method in an embodiment of the present invention is shown.

[0144] In step S1, any of the timing coding methods described in the above embodiments are used to obtain the unfolded timing value corresponding to each pixel position in the space to be tested.

[0145] In step S2, the expanded timing values ​​are substituted into the timing-height mapping formula to obtain the height data corresponding to each pixel position in the test space.

[0146] In step S3, the space to be measured is reconstructed in three dimensions based on the height data corresponding to each pixel position.

[0147] It is understood that the method for temporal coding of the test space in this application embodiment can adopt any of the temporal coding methods in the above embodiments, which will not be described again here.

[0148] In the three-dimensional reconstruction method of this invention, firstly, after setting conditions such as lighting environment, shooting environment, and line encoding parameters, multiple sets of calibration objects with known different height data are used to calibrate the calibration space to be calibrated, obtaining the temporal-height mapping relationship of the known calibration polynomial coefficients. Secondly, under the same conditions, the test space where the test object is placed is temporally encoded (i.e., the unfolded temporal value of each pixel position is obtained). The unfolded temporal value corresponding to each pixel position can then be converted into the height data corresponding to each pixel position through the temporal-height mapping relationship, thereby completing the three-dimensional reconstruction method.

[0149] In the temporal coding process of 3D reconstruction, the embodiments of the present invention improve the robustness of camera-acquired information by projecting line-coded maps, which is suitable for measurement in high dynamic range scenarios. Furthermore, when using Gray coding, binary coding can be adopted, which has high projection speed and greatly improves measurement efficiency, making it suitable for industrial applications of high reflectivity workpiece inspection.

[0150] The three-dimensional reconstruction method of this invention is based on temporal coding, which avoids the data loss problem caused by high reflection or high dynamic range scenes. It can obtain the height information of each pixel position in the space to be measured, and then obtain the number of point cloud information that matches the resolution of the captured image, so that the three-dimensional image obtained after three-dimensional reconstruction has sufficient clarity.

[0151] This invention also provides a temporal coding device for performing any of the above-described temporal coding methods for space in 3D reconstruction. For specific implementation steps, please refer to the above-described method embodiments, which will not be repeated here.

[0152] This invention also provides a three-dimensional reconstruction system for performing the above-described three-dimensional reconstruction method. The system includes:

[0153] A projection device for projecting line-coded diagrams. Furthermore, it can also be used to project Gray-coded diagrams.

[0154] An imaging device is used to capture a first image corresponding to a line-coded image. Furthermore, it can also be used to capture a second image corresponding to a Gray-coded image.

[0155] A control device is used to execute the above-described three-dimensional reconstruction method. For specific implementation steps, please refer to the above method embodiments; they will not be repeated here.

[0156] Figure 17 A schematic diagram of an electronic device for implementing a three-dimensional reconstruction method is shown in an embodiment of the present invention. The electronic device includes a memory 1100 and a processor 1200.

[0157] The processor 1200 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0158] Memory 1100 may include various types of storage units, such as system memory, read-only memory (ROM), and permanent storage devices. ROM may store static data or instructions required by processor 1200 or other modules of the computer. Permanent storage devices may be read-write storage devices. Permanent storage devices may be non-volatile storage devices that retain stored instructions and data even when the computer is powered off. In some embodiments, permanent storage devices use mass storage devices (e.g., magnetic or optical disks, flash memory) as permanent storage devices. In other embodiments, permanent storage devices may be removable storage devices (e.g., floppy disks, optical drives). System memory may be a read-write storage device or a volatile read-write storage device, such as dynamic random access memory. System memory may store some or all of the instructions and data required by the processor during operation. Furthermore, memory 1100 may include any combination of computer-readable storage media, including various types of semiconductor memory chips (DRAM, SRAM, SDRAM, flash memory, programmable read-only memory), and disks and / or optical disks may also be used. In some embodiments, memory 1100 may include a removable storage device that is readable and / or writable, such as a laser disc (CD), a read-only digital multifunction optical disc (e.g., DVD-ROM, dual-layer DVD-ROM), a read-only Blu-ray disc, an ultra-high density optical disc, a flash memory card (e.g., SD card, mini SD card, Micro-SD card, etc.), a magnetic floppy disk, etc. Computer-readable storage media do not contain carrier waves or transient electronic signals transmitted wirelessly or via wired connections.

[0159] The memory 1100 stores executable code for executing the three-dimensional reconstruction method in the above method embodiment. When the executable code is processed by the processor 1200, the processor 1200 can execute part or all of the methods described above.

[0160] In addition, the present invention provides a computer-readable storage medium, such as a chip, an optical disc, etc., on which an executable program is stored, which, when executed, implements the timing coding method or the three-dimensional reconstruction method as described in any of the preceding claims.

[0161] It should be noted that the computer-readable storage medium described in the embodiments of this disclosure is not limited to the embodiments given above. For example, it can also be an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In the embodiments of this disclosure, the computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0162] It will be understood by those skilled in the art that the above-described preferred solutions can be freely combined and superimposed without conflict. The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings; for example, two consecutively indicated blocks may actually be executed substantially in parallel, or sometimes in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions. The numbering of each step in this document is for ease of explanation and reference only and is not intended to limit the order of execution. The specific execution order is determined by the technology itself, and those skilled in the art can determine various permissible and reasonable orders based on the technology itself.

[0163] It should be noted that the use of step numbers (letters or numbers) to refer to certain specific method steps in this invention is merely for the purpose of convenience and brevity in description, and is by no means intended to restrict the order of these method steps. Those skilled in the art will understand that the order of the relevant method steps should be determined by the technology itself and should not be unduly restricted by the existence of step numbers. Those skilled in the art can determine various permissible and reasonable orderings of steps based on the technology itself.

[0164] Those skilled in the art will understand that, without conflict, the above-mentioned preferred solutions can be freely combined and superimposed.

[0165] It should be understood that the above embodiments are merely exemplary and not restrictive. Various obvious or equivalent modifications or substitutions that can be made by those skilled in the art regarding the above details without departing from the basic principles of the present invention will be included within the scope of the claims of the present invention.

Claims

1. A temporal coding method for space in three-dimensional reconstruction, characterized in that, include: N line-coded images are projected sequentially onto the space to be tested. After each line-coded image is projected, a first captured image is obtained, resulting in N first captured images and N time-series values ​​corresponding to the N first captured images. The line-coded image includes M reference lines evenly arranged along the X-axis of the projection plane. The M reference lines of the first line-coded image divide the space to be tested into M pixel intervals. The M reference lines of the next line-coded image are formed by moving the M reference lines of the previous line-coded image along the X-axis of the projection plane by a step size F. In a line-coded image, the distance between any two adjacent reference lines is F*N-1 pixels. For each pixel position in the space to be tested, based on the pixel grayscale values ​​of the same pixel position in the N first captured images, a grayscale change curve of grayscale value changing with time is fitted, and the time series value corresponding to the grayscale change peak in the grayscale change curve is determined as the folded time series value. The M pixel intervals are encoded using a preset encoding rule to obtain M encoded values, and each pixel position within a pixel interval corresponds to the same encoded value. Based on the encoding rules and the encoding values ​​corresponding to the pixel intervals where each pixel position is located, the folded timing values ​​corresponding to each pixel position are expanded to obtain the expanded timing values ​​corresponding to each pixel position in the test space. For each pixel location in the test space, based on the changes in pixel grayscale values ​​at the same pixel location in the N first captured images, a grayscale change curve is fitted, and the time series value corresponding to the grayscale change peak in the grayscale change curve is determined as the folded time series value. This includes: for any pixel location, obtaining N pixel grayscale values ​​corresponding to the pixel location in the N first captured images; fitting the N pixel grayscale values ​​into a grayscale change curve; and obtaining the time series intervals in the grayscale change curve where the pixel grayscale value is greater than a grayscale threshold. The timing value corresponding to the peak grayscale change at the pixel location is determined by combining the following formula; in, This represents the time-series value corresponding to the peak value of grayscale variation. This refers to the time interval where the pixel grayscale value is greater than the grayscale threshold. for The corresponding pixel grayscale value at the time sequence.

2. The temporal coding method for space in three-dimensional reconstruction according to claim 1, characterized in that, Before sequentially projecting N line-coded images onto the space to be tested, the process includes: The number of reference lines M in the N line-coded images is obtained based on the preset number of line-coded images N, the horizontal resolution E of the line-coded images, and the line shift step F of the line-coded images. in, F is a positive integer less than or equal to 5.

3. The temporal coding method for space in three-dimensional reconstruction according to claim 2, characterized in that, Where F=1, .

4. The temporal coding method for space in three-dimensional reconstruction according to claim 2, characterized in that, The process of encoding the M pixel intervals using a preset encoding rule includes: The number of Gray-coded maps n is set according to the number M of the pixel intervals; The period width of each gray code image is set according to the number N of the line code images, wherein the grayscale values ​​of all pixel positions within the same period width are equal; Gray coding is performed on each pixel position in the test space using n Gray code images and their corresponding period widths to mark the M pixel intervals.

5. The temporal coding method for space in three-dimensional reconstruction according to claim 4, characterized in that, The Gray code bit depth n needs to satisfy... And n takes the minimum integer value.

6. The temporal coding method for space in three-dimensional reconstruction according to claim 4, characterized in that, The step of performing Gray coding on each pixel position in the test space using n Gray code images and their corresponding period widths includes: n Gray code images are sequentially projected onto the space to be tested. Each projection of a Gray code image yields a second captured image, resulting in n second captured images. The Gray code values ​​formed by these n second captured images can identify at least M different pixel intervals. The period widths corresponding to the n Gray code images are, respectively, N, 2N, ... Or, the period widths are as follows: , , ...N.

7. The temporal coding method for space in three-dimensional reconstruction according to claim 4, characterized in that, The step of performing Gray coding on each pixel position in the test space using n Gray code images and their corresponding period widths includes: n+1+2 Gray code images are sequentially projected onto the space to be tested. Each projection of a Gray code image yields a second captured image, resulting in n+1+2 second captured images. The Gray code values ​​formed by these n+1+2 second captured images can identify at least M different pixel intervals. The period widths corresponding to the n+1 Gray code images are, respectively: 1 / 2N, N, 2N, ... Or the period widths are as follows: , , ...N,1 / 2 N; the other two Gray code diagrams are the all-black Gray code diagram and the all-white Gray code diagram.

8. The temporal coding method for space in three-dimensional reconstruction according to claim 7, characterized in that, The step of performing Gray coding on each pixel position in the test space using n Gray code images further includes: The binarization threshold of the Gray code image is determined based on the pixel grayscale values ​​of the second captured image corresponding to the all-black Gray code image and the all-white Gray code image. For any pixel location in the second captured image, if the pixel grayscale value corresponding to the pixel location is greater than the binarization threshold, then the pixel grayscale value corresponding to the pixel location is set to 1; if the pixel grayscale value corresponding to the pixel location is less than or equal to the binarization threshold, then the pixel grayscale value corresponding to the pixel location is set to 0.

9. The temporal coding method for space in three-dimensional reconstruction according to claim 4, characterized in that, Based on the Gray-coded values ​​corresponding to M pixel intervals, the folded temporal value corresponding to each pixel position is expanded, including: For any pixel location, obtain the folding timing value and Gray code value corresponding to the pixel location, and determine the pixel location using the following formula. Timing values; Where k is the pixel position Gray code value corresponding to the pixel range for The corresponding folding timing values, corresponding Timing value.

10. A method for calibrating the temporal-height mapping relationship in 3D reconstruction, characterized in that, include: Step S100: Place a calibration object with known height data in the calibration plane of the calibration space to be calibrated. The height data is the height value of each pixel position in the calibration space relative to the calibration plane. The calibration space to be calibrated has a preset temporal and height mapping relationship to be determined. The relationship includes multiple unknown calibration polynomial coefficients. Step S200: Using the timing coding method according to any one of claims 1 to 9, obtain the unfolded timing value corresponding to each pixel position in the calibration space; Step S300: Determine whether the current number of calibration measurements P is met. If not, control the calibration plane to rise a set distance and continue to execute step S200; if yes, execute step S400. Step S400: Substitute the altitude data and deployment time series values ​​corresponding to the calibration measurement of group P into the time series-altitude mapping formula to determine the unknown calibration polynomial coefficients, thereby determining the time series-altitude mapping formula.

11. The calibration method for temporal and height mapping relationships in three-dimensional reconstruction according to claim 10, characterized in that, The relational expressions include: Among them, h pixel position The corresponding height data, To calibrate the polynomial coefficients, Let be the highest order of the polynomial. pixel position The corresponding expanded timing value.

12. A three-dimensional reconstruction method, characterized in that, include: Using the timing coding method according to any one of claims 1 to 9, obtain the unfolded timing value corresponding to each pixel position in the space to be tested; Substitute the unfolded timing value into the timing-height mapping formula to obtain the height data corresponding to each pixel position in the test space; Based on the height data corresponding to each pixel position, the space to be tested is reconstructed in three dimensions.

13. The three-dimensional reconstruction method according to claim 12, characterized in that, The time-series and height mapping relationship is as follows: Among them, h pixel position The corresponding height data, The coefficients are the preset calibration polynomials. Let be the highest order of the polynomial. pixel position The corresponding expanded timing value.

14. A timing coding device, characterized in that, Used to perform the temporal coding method for space in three-dimensional reconstruction as described in any one of claims 1 to 9.

15. A three-dimensional reconstruction system, characterized in that, The system includes: Projection device for projecting line-coded diagrams; Imaging device for capturing a first image corresponding to a line-coded diagram; A control device for performing the three-dimensional reconstruction method as described in claim 12.

16. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the three-dimensional reconstruction method as described in claim 12.

17. An electronic device comprising a storage medium storing a computer program, characterized in that: When the computer program is executed by the processor, it implements the three-dimensional reconstruction method as described in claim 12.

Images