Time sequence coding method for space, three-dimensional reconstruction method and related products

Abstract

The embodiment of the invention provides a time sequence coding method for a space, a three-dimensional reconstruction method and a related product. The time sequence coding method comprises the following steps: sequentially projecting N line coding diagrams in a to-be-measured space; for each pixel position of the space to be measured, based on the pixel gray value of the same pixel position in the N first shot images, determining a time sequence value corresponding to a gray change peak value in the gray change curve as a folding time sequence value; the M pixel intervals are coded through a preset coding rule; and based on the coding rule and the coding value corresponding to the pixel interval where each pixel position is located, unfolding the folding time sequence value corresponding to each pixel position to obtain an unfolding time sequence value corresponding to each pixel position in the space to be measured. According to the embodiment of the invention, the spatial information and the time domain information are simultaneously utilized to carry out time sequence coding on each pixel position of the to-be-measured space, so that the problem of data loss during reconstruction of a high-reflection or high-dynamic region is effectively solved.

Description

Technical Field

[0001] The present invention relates to the field of three-dimensional imaging technology, and in particular to a method for temporal coding of space, a three-dimensional reconstruction method and system, a method for calibrating the relationship between temporal and height mapping, an electronic device and a computer-readable storage medium. Background Art

[0002] Machine vision is mainly used in industrial applications such as workpiece grasping, sorting, palletizing, quality inspection, assembly, and workpiece tracking. my country is a manufacturing powerhouse, and many companies need to inspect, sort, load and unload products, including a variety of highly reflective metal parts. How to use machine vision technology to correctly reconstruct the point cloud information of workpieces from a pile of scattered metal workpieces, and drive industrial robots to the correct posture to grasp workpieces and place them in the corresponding processing position through high-precision hand-eye calibration is a key technology that needs to be studied in robot grasping systems.

[0003] In the current 3D reconstruction scheme based on stripe structured light, the fringe pattern projected onto the surface of the object to be measured is analyzed, and the phase distribution of the fringe pattern is calculated using phase extraction technology. Since the height change of the surface of the object to be measured will cause the spatial distortion of the stripes, the height information of the object to be measured can be calculated by comparing the distorted stripes with the reference stripes, and then combining the relationship between the phase difference and the height change, the height information of the object to be measured can be calculated, thereby realizing 3D reconstruction. However, when facing scenes with high reflection or high dynamic areas, the fringe information is often lost due to oversaturation of the highlight area, which affects the accuracy of 3D reconstruction. Summary of the invention

[0004] Based on the above situation, the main purpose of the present invention is to provide a technical solution corresponding to a spatial time coding method, a three-dimensional reconstruction method and related products, and to simultaneously use spatial information and time domain information to perform time coding on each pixel position of the measured space, thereby effectively solving the problem of data loss during reconstruction in high-reflection or high-dynamic areas.

[0005] To achieve the above purpose, the technical solution adopted by the present invention is as follows:

[0006] A first aspect of an embodiment of the present invention provides a method for temporal coding of space in three-dimensional reconstruction, comprising: sequentially projecting N line coding images onto a space to be measured, wherein a first captured image is acquired after each projection of a line coding image, and N first captured images and N time sequence values ​​corresponding to the N first captured images are obtained; the line coding image comprises M reference lines evenly arranged along an X-axis of a projection surface, wherein the M reference lines of the first line coding image divide the space to be measured into M pixel intervals, and the M reference lines of the next line coding image are formed by moving the M reference lines of the previous line coding image along the X-axis of the projection surface by a line shift step of F pixels, and in a line coding image, the spacing between any two adjacent reference lines is F*N-1 pixels;

[0007] For each pixel position of the space to be measured, based on the pixel grayscale value of the same pixel position in the N first captured images, a grayscale change curve in which the grayscale value changes with the time sequence value is fitted, and a time sequence value corresponding to the grayscale change peak value in the grayscale change curve is determined as the folded time sequence value;

[0008] Encoding the M pixel intervals according to a preset encoding rule to obtain M encoding values, and each pixel position in a pixel interval corresponds to a same encoding value;

[0009] Based on the coding rule and the coding value corresponding to the pixel interval where each pixel position is located, the folded timing value corresponding to each pixel position is unfolded to obtain the unfolded timing value corresponding to each pixel position in the test space.

[0010] Preferably, before sequentially projecting N line coding images onto the space to be measured, the method includes:

[0011] Obtaining the number M of reference lines of the N line coding images according to the preset number N of line coding images, the lateral resolution E of the line coding images, and the line shift step F of the line coding images;

[0012] in, F is a positive integer less than or equal to 5.

[0013] Preferably, F=1,

[0014] Preferably, encoding the M pixel intervals using a preset encoding rule includes:

[0015] Setting the number n of Gray code images according to the number M of the pixel intervals;

[0016] Setting a period width of each Gray code image according to the number N of the line code images, wherein the grayscale marks of all pixel positions within the same period width are equal;

[0017] Gray coding is performed on each pixel position in the test space through n Gray code images and their corresponding period widths to mark the M pixel intervals.

[0018] Preferably, the number of Gray code bits n needs to satisfy 2 n ≥M, and n is the minimum integer value.

[0019] Preferably, the Gray coding of each pixel position in the space to be tested by using n Gray code images and their corresponding period widths includes:

[0020] n Gray code images are sequentially projected onto the space to be tested, wherein each projection of a Gray code image obtains a second captured image, and n second captured images are obtained; the Gray code values ​​formed by the n second captured images can identify at least M different pixel intervals; wherein the period widths corresponding to the n Gray code images are respectively: N, 2N, ..., 2 n-1 N, or the cycle width is: 2 n-1 NN,2 n-2 N,…N.

[0021] Preferably, the Gray coding of each pixel position in the space to be tested by using n Gray code images and their corresponding period widths includes:

[0022] n+1+2 Gray code images are sequentially projected onto the measured space, wherein each projection of a Gray code image obtains a second captured image, and n+1+2 second captured images are obtained; the Gray code values ​​formed by the n+1+2 second captured images can identify at least M different pixel intervals; wherein the period widths corresponding to the n+1 Gray code images are respectively: 1 / 2N, N, 2N, ..., 2 n-1 N; or the cycle width is: 2 n-1 N, 2 n-2 N, ...N, 1 / 2N; the other two Gray code images are a full black Gray code image and a full white Gray code image respectively.

[0023] Preferably, the Gray coding of each pixel position in the test space by using n Gray code images further includes:

[0024] Determining a binarization threshold of the Gray code image based on the pixel grayscale values ​​of the second captured image corresponding to the all-black Gray code image and the all-white Gray code image;

[0025] For any pixel position in the second captured image, if the pixel grayscale value corresponding to the pixel position is greater than the binarization threshold, the pixel grayscale value corresponding to the pixel position is set to 1; if the pixel grayscale value corresponding to the pixel position is less than or equal to the binarization threshold, the pixel grayscale value corresponding to the pixel position is set to 0.

[0026] Preferably, for each pixel position of the space to be measured, based on the change of the pixel gray value at the same pixel position in the N first captured images, fitting a gray change curve, and determining the timing value corresponding to the gray change peak in the gray change curve as the folded timing value, includes:

[0027] For any pixel position, obtain N pixel grayscale values ​​corresponding to the pixel position in the N first captured images respectively;

[0028] Fitting the N pixel grayscale values ​​into a grayscale change curve;

[0029] Obtain the time interval [t1, t2] in which the grayscale value of the pixel in the grayscale change curve is greater than the grayscale threshold, and determine the time value corresponding to the grayscale change peak value of the pixel position in combination with the following formula;

[0030]

[0031] Among them, y is the time series value corresponding to the grayscale change peak value, [t1, t2] is the time series interval where the pixel grayscale value is greater than the grayscale threshold, I t is the pixel grayscale value corresponding to the time sequence t.

[0032] Preferably, the unfolding of the folded timing value corresponding to each pixel position based on the Gray code values ​​corresponding to the M pixel intervals includes:

[0033] For any pixel position, obtain the folded timing value and Gray code value corresponding to the pixel position, and determine the unfolded timing value corresponding to the pixel position by combining the following formula: T(i, j) = kN + t(i, j);

[0034] Among them, k is the Gray code value corresponding to the pixel interval where the pixel position (i, j) is located, t(i, j) is the folded timing value corresponding to the pixel position (i, j), and T(i, j) is the unfolded timing value corresponding to the pixel position (i, j).

[0035] A second aspect of an embodiment of the present invention provides a method for calibrating a time sequence and a height mapping relationship in three-dimensional reconstruction, comprising:

[0036] Step S100: placing a calibration object with known height data in a calibration plane of the space to be calibrated, wherein the height data is the height value of each pixel position in the space to be calibrated relative to the calibration plane; wherein a timing and height mapping relationship to be determined is preset for the space to be calibrated, and the relationship includes a plurality of unknown calibration polynomial coefficients;

[0037] Step S200: using any of the time series encoding methods of the first aspect above to obtain an expanded time series value corresponding to each pixel position in the space to be calibrated;

[0038] Step S300: determining whether the preset calibration measurement times P are met at present; if not, controlling the calibration plane to rise by a set distance, and continuing to execute step S200; if yes, executing step S400;

[0039] Step S400: Substitute the height data and the expanded timing values ​​corresponding to the P groups of calibration measurements into the timing and height mapping relationship to determine the unknown calibration polynomial coefficients, thereby determining the timing and height mapping relationship.

[0040] Preferably, the relationship includes:

[0041] h(i,j)=p w T(i,j) w +p w-1 T(i,j) w-1 +…+p1T(i,j)+p0

[0042] Among them, h(i,j) is the height data corresponding to the pixel position (i,j), p0 to p w is the calibration polynomial coefficient, w is the highest order of the polynomial, and T(i, j) is the expanded time value corresponding to the pixel position (i, j).

[0043] A third aspect of an embodiment of the present invention provides a three-dimensional reconstruction method, comprising:

[0044] Using any of the timing coding methods of the first aspect above, obtaining an expanded timing value corresponding to each pixel position in the space to be measured;

[0045] Substituting the expanded timing value into a timing-height mapping relationship to obtain height data corresponding to each pixel position in the measured space;

[0046] Based on the height data corresponding to each pixel position, the space to be measured is reconstructed in three dimensions.

[0047] Preferably, the timing and height mapping relationship is:

[0048] h(i,j)=p w T(i,j) w +p w-1 T(i,j) w-1 +…+p1T(i,j)+p0

[0049] Among them, h(i,j) is the height data corresponding to the pixel position (i,j), p0 to p wis the preset calibration polynomial coefficient, w is the highest order of the polynomial, and T(i,j) is the expanded time value corresponding to the pixel position (i,j).

[0050] A fourth aspect of an embodiment of the present invention provides a time sequence coding device, which is used to execute any time sequence coding method of the first aspect above.

[0051] A fifth aspect of an embodiment of the present invention provides a three-dimensional reconstruction system, the system comprising:

[0052] A projection device for projecting a line-coded diagram;

[0053] An imaging device, used for capturing a first captured image corresponding to the line coding image;

[0054] A control device is used to execute the three-dimensional reconstruction method of the third aspect mentioned above.

[0055] A sixth aspect of an embodiment of the present invention provides a computer program stored thereon, and when the computer program is executed by a processor, the three-dimensional reconstruction method of the third aspect mentioned above is implemented.

[0056] A seventh aspect of an embodiment of the present invention provides an electronic device, comprising a storage medium storing a computer program, and when the computer program is executed by a processor, the three-dimensional reconstruction method of the third aspect is implemented.

[0057] In the 3D reconstruction scheme based on stripe structured light, it is necessary to identify the grayscale value of the pixel points corresponding to the stripes to measure the relevant height information. If only spatial information (i.e., the grayscale value corresponding to a specific pixel position) is collected during measurement, when encountering a scene with high reflection or high dynamic area, when the stripes pass through the highlight area, since the pixel points corresponding to the stripes and the pixels nearby are in a highlight saturation state (i.e., the grayscale value of the pixel points corresponding to the bright line and the grayscale values ​​of other pixels are at or close to the grayscale extreme value), the accurate position of the pixel points corresponding to the stripes cannot be effectively identified, resulting in the loss of data corresponding to these pixels.

[0058] Through the scheme provided in the above aspects of the present invention, by line coding the space to be tested, the grayscale change peak value of each pixel position in the space to be tested is associated with the corresponding timing value, and the space to be tested is divided into M pixel intervals to obtain the folded timing values ​​of the M pixel intervals, and then the folded timing values ​​are expanded by Gray coding to obtain the expanded timing values ​​corresponding to each pixel position in the space to be tested (that is, the timing coding is completed for each pixel position in the space to be tested). Since the grayscale change peak value in the embodiment of the present invention is the peak value of the grayscale change curve, and the grayscale change curve is obtained by fitting the grayscale value changes of multiple captured images at the same pixel position, that is, no matter what influence the highlight area has on the actual grayscale value of each pixel position, a grayscale change peak value can be obtained at the corresponding pixel position through simulation fitting, and then this grayscale change peak value is converted into a timing value through the grayscale change curve, so that each pixel position can obtain an expanded timing value. In the subsequent 3D reconstruction process, if 3D reconstruction is performed based on the expanded timing value of each pixel position, and the expanded timing value is converted back into height data according to the timing and height mapping relationship, the timing will not be affected by the highlight area, and the influence of the highlight area on the acquisition of height data can be avoided. This effectively solves the problem of data loss when reconstructing traditional line structured light in high-reflection or high-dynamic areas, and has significant advantages in the measurement of high-reflection or high-dynamic range scenes.

[0059] Other beneficial effects of the present invention will be explained in the specific implementation manner through the introduction of specific technical features and technical solutions. Through the introduction of these technical features and technical solutions, those skilled in the art should be able to understand the beneficial technical effects brought about by the technical features and technical solutions. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings.

[0061] Figure 1 A flow chart showing a method for temporal encoding of space in three-dimensional reconstruction according to an embodiment of the present invention is shown;

[0062] Figure 2 A schematic diagram showing a line coding diagram in an embodiment of the present invention;

[0063] Figure 3 A schematic diagram showing a 40-step line coding diagram in an embodiment of the present invention is shown;

[0064] Figure 4 A flow chart showing a method for determining a folding timing value in an embodiment of the present invention is shown;

[0065] Figure 5 A schematic diagram showing the extraction of the timing line shift center in an embodiment of the present invention is shown;

[0066] Figure 6 A schematic diagram showing a grayscale change curve in an embodiment of the present invention;

[0067] Figure 7 A schematic diagram showing an incomplete grayscale change curve in an embodiment of the present invention;

[0068] Figure 8 A schematic diagram showing a grayscale change curve after filling in an embodiment of the present invention;

[0069] Fig. 9 A schematic diagram showing a 5-bit Gray code diagram + a complementary Gray code diagram in an embodiment of the present invention is shown;

[0070] Fig.10 A schematic diagram showing the period width corresponding to a 5-bit Gray code diagram + a complementary Gray code diagram in an embodiment of the present invention;

[0071] Fig.11 A schematic diagram showing a folding sequence in an embodiment of the present invention;

[0072] Fig.12 A schematic diagram showing the unfolding sequence in an embodiment of the present invention;

[0073] Fig.13 A schematic diagram showing the folding timing of the 100th row of pixels in an embodiment of the present invention;

[0074] Fig.14 A schematic diagram showing the expansion timing of the 100th row of pixel positions in an embodiment of the present invention is shown;

[0075] Fig.15 A flow chart showing a method for calibrating a timing and height mapping relationship in an embodiment of the present invention;

[0076] Fig.16 A flow chart of a three-dimensional reconstruction method according to an embodiment of the present invention is shown;

[0077] Fig.17 A schematic diagram of an electronic device for implementing a three-dimensional reconstruction method in an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0078] The present invention is described below based on embodiments, but the present invention is not limited to these embodiments. In the following detailed description of the present invention, some specific details are described in detail. In order to avoid confusing the essence of the present invention, known methods, processes, procedures, and components are not described in detail.

[0079] In addition, persons of ordinary skill in the art will appreciate that the drawings provided herein are for illustration purposes and are not necessarily drawn to scale.

[0080] Unless the context clearly requires otherwise, throughout the specification and claims, the words "include", "comprising" and similar words should be interpreted in an inclusive sense rather than an exclusive or exhaustive sense; that is, in the sense of "including but not limited to".

[0081] In the description of the present invention, it should be understood that the terms "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance. In addition, in the description of the present invention, unless otherwise specified, "plurality" means two or more.

[0082] The specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0083] In the scheme of three-dimensional reconstruction through structured light, a projection device is used to project a set pattern of structured light onto the surface of an object in the measured space, and an imaging device cooperating with the projection device captures the structured light reflected back from the surface of the object in the measured space. Since the structured light is modulated on the surface of the object, the reflected structured light carries the surface information of the object. By demodulating the reflected structured light, the three-dimensional morphological information of the object can be obtained, and then the three-dimensional morphology of the object surface can be reconstructed.

[0084] As mentioned above, in the traditional 3D reconstruction scheme based on stripe structured light, only the spatial information of the image is collected (that is, the position information of the center of the reference line in the line structure diagram is collected). In high-reflection or high-dynamic range scenes, the image obtained by reflection may have a high-exposure area. The grayscale values ​​of the pixels in this area reach or are close to the grayscale extreme value, and the edge information of the object cannot be effectively distinguished, resulting in data loss during 3D reconstruction.

[0085] In view of this, an embodiment of the present invention proposes a method for temporal encoding of space in three-dimensional reconstruction, and simultaneously utilizes spatial information and time domain information to perform temporal encoding on each pixel position of the measured space, which can avoid the influence of highlight areas on the acquisition of height data and effectively solve the problem of data loss during reconstruction in high-reflection or high-dynamic areas.

[0086] See also Figure 1 , Figure 1 A flow chart of a method for temporal coding of space in three-dimensional reconstruction in an embodiment of the present invention is shown. An embodiment of the present invention provides a method for temporal coding of space in three-dimensional reconstruction, which includes steps S10 to S40.

[0087] In step S10, N line coding images are sequentially projected onto the space to be measured, wherein each time a line coding image is projected, a first captured image is obtained, and N first captured images are obtained. Moreover, each time a first captured image is captured, a corresponding timing value is recorded (optionally, after completing N first captured images, the timing value corresponding to each first captured image can be uniformly calculated, or the shooting timing can be recorded in real time. The specific requirements can be determined according to actual needs and are not limited here). Therefore, N first captured images are recorded to obtain N timing values ​​(t=1, 2, ..., N-1, N). It should be noted that in the embodiment of the present invention, for the convenience of explanation and illustration, the timing value uses a numerical sequence as an example. In actual applications, the timing value can correspond to an actual time value, such as t1=8:01:01, t2=8:01:02...tN=8:01:0N. The timing interval between the specific N timing values ​​depends on the shooting frequency of the imaging device, which is not limited here.

[0088] Figure 2 FIG. 2 is a schematic diagram showing a line coding diagram in an embodiment of the present invention. Figure 2 As shown, the line coding diagram includes M reference lines evenly arranged along the X-axis of the projection surface (ie, Figure 2 ), wherein the width of each reference line is one pixel. Specifically, in a line coding image, the spacing between any two adjacent reference lines is F*N-1 pixels. For N consecutive line coding images, the reference lines of any two adjacent line coding images are staggered by the width of F pixels on the X-axis of the projection surface. Wherein, F represents the line shift step, which is a positive integer less than or equal to 5. For a group of N consecutive line coding images, under continuous projection, the reference lines look like they are moving along the X-axis. Therefore, a group of continuous line coding images is also called a group of line shift images. Wherein, F is the line shift step (that is, the distance the reference line moves on the X-axis each time), and N is the number of line shift steps. Exemplarily, the first reference line of the first line coding image is Figure 2 The bright line corresponding to the first pixel point on the X axis, if the projection resolution set by the line coding map is E, then (That is, the number of reference lines is equal to the result of dividing the lateral resolution by (the line shift step * the number of line coding images) and rounding down). It can be seen that the M reference lines of the first line coding image divide the space to be tested into M pixel areas with a width of F*N. It should be noted that in the embodiment of the present invention, N is not equal to E, and the minimum value of M is 2, that is, the space to be tested is divided into at least two pixel intervals.

[0089] For example, assuming that the projection resolution of the line coding image is 1280*720 pixels, the number of line coding images N=40 and the line shift step F is 1 pixel, Figure 3 A schematic diagram of a 40-step line coding diagram in an embodiment of the present invention is shown. Figure 3As shown, the distance between any two adjacent reference lines in each line coding image is 39 pixels. The pattern of the first line coding image can be referred to Figure 2 , each bright line (reference line) in the figure occupies 1 pixel in the X-axis direction. The first bright line position of the first line coding image is the first column. The pixel value of this column is set to 1 (0 is black and 1 is white in a binary image). Then, the next bright line is generated repeatedly according to the line interval period until there are 32 (1280÷40=32) bright lines on the image. Then, the first line coding image is generated. The subsequent line coding images are generated by moving each bright line backward by 1 pixel (F=1); and so on, until 40 line coding images are obtained. Among them, the j-th line coding image can be expressed by the following formula:

[0090]

[0091] Wherein, j=1, 2, ..., N, u and v are the horizontal and vertical coordinates of the image respectively, N is the number of line coding images, and % represents the modulo operation.

[0092] In step S20, for each pixel position in the space to be tested, a grayscale change curve in which the grayscale value changes with the time sequence value is fitted based on the pixel grayscale value at the same pixel position in the N first captured images, and the time sequence value corresponding to the grayscale change peak in the grayscale change curve is determined as the folded time sequence value.

[0093] In an implementation manner of step S20, steps S21 to S25 may be included. Figure 4 , Figure 4 A flow chart of a method for determining a folding timing value in an embodiment of the present invention is shown.

[0094] In step S21, a pixel position is selected, and N pixel grayscale values ​​corresponding to the pixel position in the N first captured images are obtained.

[0095] In step S22, the grayscale values ​​of N pixels are fitted into a grayscale variation curve.

[0096] In step S23, the time interval [t1, t2] in which the pixel grayscale value in the grayscale change curve is greater than the grayscale threshold is obtained, and the time value corresponding to the grayscale change peak value of the current pixel position is determined in combination with the following grayscale centroid method formula.

[0097]

[0098] Among them, y is the time series value corresponding to the grayscale change peak value, [t1, t2] is the time series interval where the pixel grayscale value is greater than the grayscale threshold, I t is the pixel grayscale value corresponding to the time sequence t. Figure 5 A schematic diagram showing the extraction of the timing line shift center in an embodiment of the present invention is shown as follows: Figure 5 As shown, the I indicated in the figure th is the grayscale threshold.

[0099] The grayscale threshold is a lower limit set for valid data, that is, only data with a pixel grayscale value greater than the grayscale threshold is selected for subsequent algorithm processing.

[0100] In step S24, it is determined whether there are any unselected pixel positions in the current test space. If yes, the process returns to step S21; if not, the process proceeds to step S25.

[0101] In step S25, the folding timing value corresponding to each pixel position in the test space is output.

[0102] For example, assuming that the projection resolution of the line coding image is 1280*720 pixels, the number of line coding images is 40 and the line shift step is 1 pixel. The gray value curve distribution of a pixel position at 40 time sequences (t=1, 2, ..., 39 and 40) is as follows: Figure 6 As shown, Figure 6 A schematic diagram showing a grayscale change curve in an embodiment of the present invention. Figure 6 At a certain time t, it does not obviously reach the peak value, or does not completely conform to the Gaussian curve distribution (that is, at this time, it is possible that the reference line passes through a highlight area, and the grayscale value of the pixel corresponding to the reference line cannot be effectively identified). At this time, the formula of the grayscale centroid method can be used to perform weighted calculation of the time series center value, and simulate and calculate at a certain time series t (for example, t=16.25) that the pixel grayscale value reaches the grayscale change peak (that is, at what time series t is the corresponding pixel grayscale value reaching the maximum).

[0103] In other scenarios of the embodiments of the present invention, due to the position differences of each pixel, the grayscale value distribution of some pixels does not completely conform to the Gaussian distribution. Figure 7 As shown, Figure 7 A schematic diagram of an incomplete grayscale change curve in an embodiment of the present invention is shown. In the embodiment of the present invention, what is calculated is the moment when the grayscale value of a certain pixel reaches the maximum. Since the grayscale change curve in the embodiment of the present invention is obtained by fitting N discrete pixel grayscale values, the grayscale change peak may not appear in the N pixel grayscale values. If a grayscale change curve that conforms to or approximates a Gaussian distribution cannot be fitted, the timing value corresponding to the obtained grayscale change peak may not be accurate enough.

[0104] For the pixels whose gray value distribution does not meet the Gaussian distribution standard, the latter section and the former section of the current gray value change curve can be intercepted ( Figure 7The image of the area framed by the dotted line in the middle is simulated to fill the front area and the back area ( Figure 8 The area framed by the dashed line in the figure is simulated as follows Figure 8 The grayscale change curve shown is Figure 8 A schematic diagram of a grayscale change curve after filling in an embodiment of the present invention is shown, and the grayscale change curve after filling is used to obtain the timing value corresponding to the grayscale change peak value. Figures 6 to 8 In the schematic diagram, the horizontal axis corresponds to the timing value, and the vertical axis corresponds to the grayscale value. Figure 7 In the case of , the corresponding discrete data in the time series interval [t1, t2] can also be substituted into the formula of the grayscale centroid method for calculation, but due to Figure 7 The continuity of the data is not good, so the results are not accurate enough. Figure 7 The data is intercepted and fitted to obtain Figure 8 , and then substitute Figure 8 The continuous data in is calculated to obtain a more accurate timing value corresponding to the grayscale change peak.

[0105] In the above scheme, the grayscale change peak value determined corresponds to the position of the reference line in the line coding image (the grayscale value of the bright line is the largest). In the process of line shifting of N line coding images, the corresponding N first captured images have a nearly Gaussian distribution. Therefore, the weighted center position of the grayscale value is extracted according to the above grayscale centroid method, that is, the time value corresponding to the pixel position.

[0106] Regarding the concept of folded timing values, as described above, each time a line coding image is projected, a first captured image is captured, and a timing value is correspondingly recorded. Assuming that the step width of the line shift is 1 pixel (i.e., F=1), after taking N consecutive photos, if the first reference line of each line coding image is used as the observation angle of the line shift, the first reference line is equivalent to traversing the first pixel area of ​​the space to be measured, and N timing values ​​are recorded during the traversal process; if the second reference line of each line coding image is used as the observation angle of the line shift, the second reference line is equivalent to traversing the second pixel area of ​​the space to be measured, and N timing values ​​are also recorded during the traversal process; the subsequent Mth reference line is analogous. Please note that in this process, the N timing values ​​recorded during the traversal of each reference line are the same (e.g., the timing values ​​of the first reference line during traversal are t=1, 2, ..., N-1 and N; and the timing values ​​of the second reference line during traversal are also t=1, 2, ..., N-1 and N), which is equivalent to folding M pixel intervals into one pixel interval in terms of timing. Therefore, the timing value corresponding to the grayscale change peak at this time is the folded timing value.

[0107] In other scenarios of the embodiments of the present invention, the stride of the line shift can be set according to the actual computing requirements. For example, assuming that the stride of the line shift is 2 pixels (i.e., F=2), it is equivalent to sampling the original process of traversing all pixels by half. Although some accuracy is sacrificed, the gray value data obtained by sampling can also be fitted into a gray change curve. The final time series coding result will not have too much deviation, but the data processing overhead is reduced, and the efficiency of time series coding is improved.

[0108] In step S30, the M pixel intervals are encoded according to a preset encoding rule to obtain M encoding values, and each pixel position in a pixel interval corresponds to a same encoding value.

[0109] As mentioned above, since the space to be tested is divided into M pixel intervals, and these M pixel intervals are folded into one pixel interval in time sequence, it is necessary to sequentially expand the folded timing values ​​corresponding to each pixel interval in order to obtain the accurate timing values ​​corresponding to each pixel position in the space to be tested. Therefore, before performing the expansion operation, it is necessary to encode these M pixel intervals in order to obtain the encoding values ​​corresponding to the M pixel intervals respectively. When performing the time sequence expansion, the M encoding values ​​are sequentially decoded according to the preset encoding rules, so as to combine the decoding results with the corresponding folded timing values ​​to calculate the corresponding expanded timing values. It should be noted that any encoding rule can be used as the preset encoding rule in the embodiment of the present invention as long as it can encode the pixel intervals in order (and can reversely obtain the corresponding encoding sequence according to the original encoding rule).

[0110] In step S40, based on the coding rule and the coding value corresponding to the pixel interval where each pixel position is located, the folded timing value corresponding to each pixel position is expanded to obtain the expanded timing value corresponding to each pixel position in the test space.

[0111] For example, see Fig.11 , Fig.12 , Fig.13 and Fig.14 . Fig.11 A schematic diagram showing the folding sequence in an embodiment of the present invention. Fig.13 A schematic diagram of the folding timing of the 100th row of pixels in an embodiment of the present invention is shown. Fig.12 A schematic diagram showing the unfolding sequence in an embodiment of the present invention. Fig.14 The following is a schematic diagram of the unfolding sequence of the pixel position of the 100th row in an embodiment of the present invention. Assuming that the projection resolution of the line coding image is 1280*720 pixels, the resolution of the captured image is 2048*1376 pixels, the number of line coding images is 40, and the line shift step is 1 pixel, the folding sequence image without level unfolding is as follows: Fig.11As shown, accordingly, the expanded time sequence image after time sequence expansion is as follows Fig.13 As shown. The folding time sequence value corresponding to the horizontal pixel point at the 100th row of the captured image is as follows Fig.12 As shown in the figure (the ordinate in the figure represents the timing value, and the abscissa represents the pixel point), accordingly, after the level expansion, the expansion timing value corresponding to the horizontal pixel point at the 100th row position is as follows: Fig.14 shown.

[0112] In the above step S30, Gray coding can be used as a preset coding rule. In the three-dimensional reconstruction based on structured light, Gray coding is a kind of coding used to assist phase unwrapping. Among them, there is only one bit difference between any two adjacent Gray code values ​​(that is, at most one bit error is generated in a set of Gray code values), so it has strong anti-interference ability, simple coding, and a wide range of application.

[0113] Specifically, Gray coding can be performed on each pixel position in the test space by using n Gray code images, and the test space is marked as M pixel intervals by the n Gray code images. Exemplarily, before Gray coding is performed, the number of bits n required for Gray coding can be determined according to the number of line coding images N of the line coding image and the set lateral resolution E. The set lateral resolution refers to the lateral resolution of the preset line coding image or Gray code image. The number of Gray code bits n needs to satisfy 2 n ≥M, and n takes the minimum integer value.

[0114] For example, in one implementation of step S30, n Gray code images may be sequentially projected onto the space to be tested, wherein each projection of a Gray code image acquires a second captured image, thereby obtaining n second captured images; the Gray code values ​​formed by the n second captured images can identify at least M different pixel intervals. The period widths corresponding to the n Gray code images are respectively: N, 2N, ..., 2 n-1 N, or the cycle width is: 2 n-1 N, 2 n-2 N,…N.

[0115] Exemplarily, in the embodiment of the present invention, all pixel positions in the space to be tested are gray-coded. Fig. 9 , Fig. 9 The schematic diagram of the 5-bit Gray code image + complementary Gray code image in the embodiment of the present invention is shown, GC1, GC2, GC3, GC4 and GC5 in the figure represent the first to fifth Gray code images respectively, and GC6 is a complementary Gray code image. Among them, the five Gray code images GC1, GC2, GC3, GC4 and GC5 can divide the horizontal pixels into 32 pixel intervals (n is the number of Gray code bits, that is, 2 nEach Gray code diagram represents a certain bit of Gray code, black in the diagram represents 0, and white represents 1.

[0116] In an embodiment of the present invention, the cycle width represents the length of a Gray code image in which the Gray code value does not change continuously (that is, the grayscale marks of all pixel positions within the same cycle width are equal. Among them, the grayscale mark can be marked with "0" and "1" respectively, and in actual applications, the grayscale mark "0" does not mean that the grayscale value must be 0, and may indicate that the grayscale value corresponding to the pixel is lower than the binarization threshold; and similarly, the grayscale mark "1" may indicate that the grayscale value corresponding to the pixel is higher than the binarization threshold). It should be noted that the method of performing Gray encoding by projecting a Gray code image is only an example. It can be understood that in actual applications, there may be other methods to implement Gray encoding (such as simulating the captured image of the Gray code image in the test space by superimposing layers in a computer), which is not specifically limited here.

[0117] In another implementation of step S30, since edge jumps may occur when Gray coding is expanded in stages, complementary Gray codes may be introduced during Gray coding to improve the stability and accuracy of decoding. Specifically, n+1+2 Gray code images may be projected sequentially onto the space to be tested, wherein each projection of a Gray code image obtains a second captured image, and n+1+2 second captured images are obtained; the Gray code values ​​formed by the n+1+2 second captured images can identify at least M different pixel intervals; wherein the period widths corresponding to the n+1 Gray code images are respectively: 1 / 2N, N, 2N, ..., 2 n-1 N; or the cycle width is: 2 n-1 N, 2 n-2 N, ...N, 1 / 2N; the other two Gray code images are a full black Gray code image and a full white Gray code image respectively.

[0118] In another embodiment of step S30, in addition to being used to set the coding bit of the corresponding pixel point, the Gray code map can also be used to assist in marking the pixel grayscale value corresponding to the second captured image. Specifically, n+2 Gray code maps can be projected sequentially onto the measured space, wherein each projection of a Gray code map obtains a second captured image, and obtains n+2 second captured images; the Gray code values ​​formed by the n+2 second captured images can identify at least M different pixel intervals. Among them, the period widths corresponding to the n Gray code maps are respectively: N, 2N, ..., 2 n-1 N, or the cycle width is: 2 n-1 N, 2 n-2 N, ...N; the other two Gray code images are a full black Gray code image and a full white Gray code image respectively.

[0119] Exemplarily, the binarization threshold of the second captured image can be determined based on the pixel grayscale values ​​of the second captured image corresponding to the all-black Gray code image and the all-white Gray code image. For example, the second captured image corresponding to the all-black Gray code image and the all-white Gray code image is collected by an imaging device, and the binarization threshold I is calculated based on the following formula: threshold :

[0120]

[0121] Among them, I w is the pixel grayscale value of the second captured image corresponding to the all-white Gray code image, I b is the pixel grayscale value of the second captured image corresponding to the all-black Gray code image. For any pixel position in the second captured image, if the pixel grayscale value corresponding to the pixel position is greater than the binarization threshold, the pixel grayscale value corresponding to the pixel position is set to 1; if the pixel grayscale value corresponding to the pixel position is less than or equal to the binarization threshold, the pixel grayscale value corresponding to the pixel position is set to 0.

[0122] For example, it is assumed that the period widths corresponding to the n+1 Gray code images are respectively: 2 n-1 N, 2 n-2 N, ...N, 1 / 2N, then the Gray code corresponding to the period width of 1 / 2N is the complementary Gray code. Fig.10 , Fig.10 A schematic diagram showing the period widths corresponding to the "5+1" Gray code diagrams in the embodiment of the present invention. Wherein, k1 represents the coding value (decimal) corresponding to the Gray code diagram, k2 represents the coding value (decimal) corresponding to the complementary Gray code diagram, Fig.10 GC6 in the figure represents the cycle width corresponding to the complementary Gray code image. Assuming that the projection resolution of the line coding image is 1280*720 pixels, the number of line coding images is 40 and the line shift step is 1 pixel, the different pixel intervals that need to be identified by Gray coding are 32, n=5, and one more complementary Gray code image is added. The cycle widths corresponding to these "5+1" Gray code images are: 640 (pixels), 320, 160, 80, 40 and 20 respectively. When calculating the expansion timing value according to the Gray code, the following complementary Gray code formula can be used for processing.

[0123]

[0124] Among them, t(i, j) represents the folded timing value, T(i, j) represents the unfolded timing value, k1 represents the encoding value (decimal) corresponding to the Gray code diagram, and k2 represents the encoding value (decimal) corresponding to the complementary Gray code diagram. It can be seen from the above formula that when the folded timing value is in the interval of t(i, j)≤-(F*N) / 4, or t(i, j)≥(F*N) / 4, this interval is a position where edge jumps are prone to occur. At this time, complementary Gray code is used for decoding. As long as the offset of the Gray code does not exceed (F*N) / 4, the accurate timing value can be restored by complementary Gray code.

[0125] Specifically, in another implementation of step S40, for any pixel position, the folded timing value and Gray code value corresponding to the pixel position are obtained, and the unfolded timing value corresponding to the pixel position is determined in combination with the following formula.

[0126] T(i,j)=kN+t(i,j)

[0127] Wherein, k is the Gray code value corresponding to the pixel interval where the pixel position (i, j) is located, t(i, j) is the folding timing value corresponding to the pixel position (i, j), and T(i, j) is the expansion timing value corresponding to the pixel position (i, j). In practical applications, when the Gray code is expanded in stages, edge jumps may occur. Therefore, the embodiment of the present invention sets GC6 as a complementary Gray code diagram. When the decoding area is located at the edge of the pixel area, the complementary Gray code value k2 is used for expansion; when the decoding area is located at the non-edge position of the pixel area, the normal Gray code value k1 is used for expansion. Specifically, please refer to the above complementary Gray code formula. If the folding timing value is in the interval of t(i, j)≤-(F*N) / 4, or t(i, j)≥(F*N) / 4, then T(i, j)=t(i, j)+k2*F*N, or T(i, j)=t(i, j)+(k2-1)*F*N when expanded. If the folding timing value is in If it is within the interval, then T(i, j)=t(i, j)+k1*F*N when expanded.

[0128] In the temporal coding method of space in three-dimensional reconstruction in an embodiment of the present invention, by line coding the space to be tested, the grayscale change peak value of each pixel position in the space to be tested is associated with the corresponding timing value, and the space to be tested is divided into M pixel intervals to obtain folded timing values ​​of the M pixel intervals, and then the folded timing values ​​are expanded by Gray coding to obtain the expanded timing values ​​corresponding to each pixel position in the space to be tested (that is, the timing coding is completed for each pixel position in the space to be tested).

[0129] First, since the grayscale change peak in the embodiment of the present invention is the peak of the grayscale change curve, and the grayscale change curve is obtained by fitting the grayscale value changes of multiple captured images at the same pixel position, that is, no matter what influence the highlight area has on the actual grayscale value of each pixel position, a grayscale change peak can be obtained by simulation fitting at the corresponding pixel position, and then the grayscale change peak is converted into a timing value through the grayscale change curve, so that each pixel position can obtain an expanded timing value. In the subsequent three-dimensional reconstruction process, if three-dimensional reconstruction is performed based on the expanded timing value of each pixel position, and the expanded timing value is converted back to height data according to the timing and height mapping relationship, since the timing will not be affected by the highlight area, the influence of the highlight area on the acquisition of height data can be avoided, thereby effectively solving the problem of data loss when reconstructing in high-reflection or high-dynamic areas with traditional line structured light, and has significant advantages in the measurement of high-reflection or high-dynamic range scenes.

[0130] Secondly, the embodiment of the present invention divides the space to be tested into multiple pixel intervals by using a line coding map including multiple reference lines to perform line coding on the space to be tested, and associates the grayscale change peak value of each pixel position in each pixel interval with the corresponding timing value, so that multiple pixel intervals can be folded into one pixel interval in time sequence, and then the pixel intervals can be encoded and then expanded in sequence, thereby reducing the demand for line coding maps by folding pixel intervals, thereby greatly reducing the requirements for hardware performance, and greatly enhancing processing efficiency and product performance.

[0131] Furthermore, both the line coding and Gray coding in the embodiments of the present invention can be quickly encoded through projection, and compared with the structured light formed by traditional laser scanning, the data processing efficiency is higher.

[0132] In the scheme of three-dimensional reconstruction by line structured light in the embodiment of the present invention, since the light is modulated differently by surfaces at different heights, in the consecutive N first captured images, for any same pixel position, the timing value corresponding to the grayscale change peak value is different at different heights. For example, when other conditions are the same, for the height variable of any pixel position, the higher the height data of the pixel position, the earlier the grayscale change peak value is reached on the timing axis corresponding to the pixel position. Therefore, a correlation between height change and timing change can be established through prior calibration, so that after the measured space is subsequently time-coded, the height data of each pixel position can be converted through the timing and height mapping relationship obtained by calibration.

[0133] In actual applications, the modulation of light by different object surfaces is a complex process, which involves multiple phenomena such as reflection, refraction and scattering of light. These phenomena work together to cause light to be modulated when passing through different object surfaces, that is, the intensity, phase, polarization state and reflection time of light change. Therefore, before measuring the height data of an object, multiple sets of calibration objects with known different height data can be used (the calibration objects and the object to be measured can also be the same or similar materials). Under the same environment (such as light environment and temperature and humidity environment), the calibration space is subjected to the same line encoding (that is, the parameters such as N, M and F are consistent) to eliminate interference from various factors other than the height data, so that the timing value corresponding to the grayscale change peak value of each pixel position in the calibration space can reflect the change in height data, forming an accurate mapping relationship between timing and height mapping. Specifically, an embodiment of the present invention also provides a calibration method for the timing and height mapping relationship in three-dimensional reconstruction, which includes steps S100 to S400. Please refer to Fig.15 , Fig.15 A flow chart showing a method for calibrating a timing and height mapping relationship in an embodiment of the present invention.

[0134] In step S100, a calibration object with known height data is placed in the calibration plane of the space to be calibrated. In the embodiment of the present invention, the space to be measured refers to the space where the object to be measured is placed, and the calibration space refers to the space calibrated by the calibration object. The height data is the height value of each pixel position in the space to be calibrated relative to the calibration plane. In addition, for the space to be calibrated, a timing and height mapping relationship to be determined is preset, and the relationship includes multiple unknown calibration polynomial coefficients.

[0135] Specifically, the calibration plane can be controlled by a displacement mechanism to perform precise displacement in the height direction. After each calibration measurement of the current height is completed, it will move a set distance in the same direction and continue with the next set of calibration measurements. For example, in order to obtain an accurate mapping relationship between the timing and height of the current space, 10 calibration measurements can be performed, each moving 1 mm.

[0136] In step S200, any one of the time series encoding methods described in the above embodiments is used to obtain the expanded time series value corresponding to each pixel position in the space to be calibrated.

[0137] In step S300, it is determined whether the preset calibration measurement times P are met. If not, the calibration plane is controlled to rise by a set distance and step S200 is continued; if yes, step S400 is executed.

[0138] In step S400, the height data and the expanded time series value corresponding to the calibration measurement of group P are substituted into the time series and height mapping relationship to determine the corresponding calibration polynomial coefficients. Among them, the height data of step S400 is the height value of each pixel position in the space to be calibrated relative to the calibration plane during the calibration measurement. The height data can be calculated accordingly based on the height information of the calibration object and the current height value of the calibration plane.

[0139] Specifically, the above timing and height mapping relationship can be:

[0140] h(i,j)=p w T(i,j) w +p w-1 T(i,j) w-1 +…+p1T(i,j)+p0

[0141] In step S400, h(i,j) is the known height data of the pixel position (i,j), and p0 to p w is the unknown calibration polynomial coefficient, w is the highest order of the polynomial, and T(i, j) is the expanded timing value corresponding to the pixel position (i, j). In the embodiment of the present invention, the height data and expanded timing value obtained by the calibration measurement of group P are respectively substituted into the timing and height mapping relationship, and the solution can be obtained from o0 to p w The specific value of w+1 is obtained from the known values ​​of p0 to p w The timing and height mapping relationship.

[0142] It can be understood that the method for time-series encoding the space to be calibrated in the embodiment of the present application can adopt any time-series encoding method in the above embodiments, which will not be repeated here.

[0143] In an embodiment of the present invention, after the time sequence and height mapping relationship of the measured space is calibrated, the height data corresponding to each pixel position in the measured space can be obtained by performing time sequence encoding on the measured space. Specifically, a three-dimensional reconstruction method in an embodiment of the present invention includes steps S1 to S3. Fig.16 , Fig.16 A flow chart of a three-dimensional reconstruction method in an embodiment of the present invention is shown.

[0144] In step S1, any one of the time sequence encoding methods described in the above embodiments is used to obtain the expanded time sequence value corresponding to each pixel position in the measured space.

[0145] In step S2, the expanded timing value is substituted into the timing and height mapping relationship to obtain the height data corresponding to each pixel position in the measured space.

[0146] In step S3, the space to be measured is three-dimensionally reconstructed based on the height data corresponding to each pixel position.

[0147] It can be understood that the method for time-series coding the space to be measured in the embodiment of the present application can adopt any time-series coding method in the above embodiments, which will not be described in detail here.

[0148] In the three-dimensional reconstruction method of the embodiment of the present invention, first, after setting the conditions such as the light environment, the shooting environment and the line coding parameters, the space to be calibrated is calibrated using multiple sets of calibration objects with known different height data, and the time-to-height mapping relationship of the known calibration polynomial coefficients is obtained. Secondly, under the same conditions, the space to be measured where the object to be measured is placed is time-coded (that is, the expanded time-to-height value of each pixel position is obtained), and the expanded time-to-height mapping relationship can be used to convert the expanded time-to-height value corresponding to each pixel position into the height data corresponding to each pixel position, thereby completing the three-dimensional reconstruction method.

[0149] In the temporal coding process of three-dimensional reconstruction, the embodiment of the present invention improves the robustness of camera information acquisition by projecting line coding images, which is suitable for the measurement of high dynamic range scenes; and, in Gray coding, a binary coding method can be used, with high projection speed and greatly improved measurement efficiency, which is suitable for industrial applications of high-reflective workpiece detection.

[0150] The three-dimensional reconstruction method of the embodiment of the present invention is implemented based on time coding, which avoids the problem of data loss caused by high reflection or high dynamic range scenes. The height information of each pixel position in the measured space can be obtained, and then the number of point cloud information matching the resolution of the captured image can be obtained, so that the three-dimensional image obtained after three-dimensional reconstruction has sufficient clarity.

[0151] An embodiment of the present invention further provides a time coding device for executing any of the above-mentioned time coding methods for space in three-dimensional reconstruction. For specific implementation steps, please refer to the above-mentioned method embodiments, which will not be repeated here.

[0152] The embodiment of the present invention further provides a 3D reconstruction system for executing the above 3D reconstruction method, the system comprising:

[0153] The projection device is used to project a line coding diagram. Further, it can also be used to project a Gray coding diagram.

[0154] The imaging device is used to capture a first captured image corresponding to a line coding image, and further, can be used to capture a second captured image corresponding to a Gray coding image.

[0155] The control device is used to execute the above three-dimensional reconstruction method. For specific implementation steps, please refer to the above method embodiment, which will not be repeated here.

[0156] Fig.17 A schematic diagram of an electronic device for implementing a three-dimensional reconstruction method in an embodiment of the present invention is shown. The electronic device includes: a memory 1100 and a processor 1200.

[0157] The processor 1200 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc.

[0158] The memory 1100 may include various types of storage units, such as system memory, read-only memory (ROM), and permanent storage devices. Among them, ROM can store static data or instructions required by the processor 1200 or other modules of the computer. The permanent storage device may be a readable and writable storage device. The permanent storage device may be a non-volatile storage device that does not lose the stored instructions and data even after the computer is powered off. In some embodiments, the permanent storage device uses a large-capacity storage device (such as a magnetic or optical disk, flash memory) as a permanent storage device. In some other embodiments, the permanent storage device may be a removable storage device (such as a floppy disk, optical drive). The system memory may be a readable and writable storage device or a volatile readable and writable storage device, such as a dynamic random access memory. The system memory may store some or all instructions and data required by the processor at run time. In addition, the memory 1100 may include any combination of computer-readable storage media, including various types of semiconductor memory chips (DRAM, SRAM, SDRAM, flash memory, programmable read-only memory), and disks and / or optical disks may also be used. In some embodiments, the memory 1100 may include a readable and / or writable removable storage device, such as a laser disc (CD), a read-only digital versatile disc (e.g., DVD-ROM, double-layer DVD-ROM), a read-only Blu-ray disc, an ultra-density optical disc, a flash memory card (e.g., SD card, mini SD card, Micro-SD card, etc.), a magnetic floppy disk, etc. The computer-readable storage medium does not include carrier waves and transient electronic signals transmitted wirelessly or wired.

[0159] The memory 1100 stores executable codes for executing the three-dimensional reconstruction method in the above method embodiment. When the executable codes are processed by the processor 1200, the processor 1200 can execute part or all of the above method.

[0160] In addition, the present invention also provides a computer-readable storage medium, such as a chip, a CD, etc., on which an execution program is stored. When the execution program is executed, the timing encoding method or three-dimensional reconstruction method as described in any one of the above items is implemented.

[0161] It should be noted that the computer-readable storage medium described in the embodiments of the present disclosure is not limited to the above-mentioned embodiments, and may also be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the embodiments of the present disclosure, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, device or device.

[0162] It will be appreciated by those skilled in the art that, under the premise of no conflict, the above-mentioned preferred solutions can be freely combined and superimposed. Among them, the flowcharts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, a program segment, or a part of a code, and the module, program segment, or a part of a code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings, for example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions. The numbering of each step in this article is only for the convenience of description and reference, and is not used to limit the order of execution. The specific execution order is determined by the technology itself, and those skilled in the art can determine various allowable and reasonable orders based on the technology itself.

[0163] It should be noted that the use of step numbers (letters or numbers) to refer to certain specific method steps in the present invention is only for the purpose of convenience and brevity of description, and is by no means intended to limit the order of these method steps by letters or numbers. Those skilled in the art will understand that the order of the relevant method steps should be determined by the technology itself and should not be inappropriately limited by the existence of step numbers. Those skilled in the art can determine various permissible and reasonable step orders based on the technology itself.

[0164] Those skilled in the art will appreciate that, without conflict, the above-mentioned preferred solutions can be freely combined and superimposed.

[0165] It should be understood that the above-mentioned embodiments are merely illustrative and not restrictive. Without departing from the basic principles of the present invention, various obvious or equivalent modifications or substitutions that can be made by those skilled in the art to the above-mentioned details will all be included in the scope of the claims of the present invention.

Claims

1. A method for temporal coding of space in three-dimensional reconstruction, characterized in that: include: N line coding images are sequentially projected onto the space to be tested, wherein a first captured image is acquired after each projection of a line coding image, and N first captured images and N time sequence values ​​corresponding to the N first captured images are obtained; the line coding image includes M reference lines evenly arranged along the X-axis of the projection surface, wherein the M reference lines of the first line coding image divide the space to be tested into M pixel intervals, and the M reference lines of the next line coding image are formed by moving the M reference lines of the previous line coding image along the X-axis of the projection surface with a line shift step of F pixels, and in a line coding image, the spacing between any two adjacent reference lines is F*N-1 pixels; For each pixel position of the space to be measured, based on the pixel grayscale value of the same pixel position in the N first captured images, a grayscale change curve in which the grayscale value changes with the time sequence value is fitted, and a time sequence value corresponding to the grayscale change peak value in the grayscale change curve is determined as the folded time sequence value; Encoding the M pixel intervals according to a preset encoding rule to obtain M encoding values, and each pixel position in a pixel interval corresponds to a same encoding value; Based on the coding rule and the coding value corresponding to the pixel interval where each pixel position is located, the folded timing value corresponding to each pixel position is unfolded to obtain the unfolded timing value corresponding to each pixel position in the test space.

2. The method for temporal encoding of space in three-dimensional reconstruction according to claim 1, characterized in that: Before sequentially projecting N line coding images into the space to be measured, the method includes: Obtaining the number M of reference lines of the N line coding images according to the preset number N of line coding images, the lateral resolution E of the line coding images, and the line shift step F of the line coding images; in, F is a positive integer less than or equal to 5.

3. The method for temporal encoding of space in three-dimensional reconstruction according to claim 2, characterized in that: Where F = 1, 4. The method for temporal encoding of space in three-dimensional reconstruction according to claim 2, characterized in that: The encoding of the M pixel intervals by using a preset encoding rule includes: Setting the number n of Gray code images according to the number M of the pixel intervals; Setting a period width of each Gray code image according to the number N of the line code images, wherein the grayscale marks of all pixel positions within the same period width are equal; Gray coding is performed on each pixel position in the test space through n Gray code images and their corresponding period widths to mark the M pixel intervals.

5. The method for temporal encoding of space in three-dimensional reconstruction according to claim 4, characterized in that: The number of Gray code bits n needs to satisfy 2 n ≥M, and n is the minimum integer value.

6. The method for temporal encoding of space in three-dimensional reconstruction according to claim 4, characterized in that: The Gray coding of each pixel position in the space to be tested by using n Gray code images and their corresponding period widths includes: n Gray code images are sequentially projected onto the space to be tested, wherein each projection of a Gray code image obtains a second captured image, and n second captured images are obtained; the Gray code values ​​formed by the n second captured images can identify at least M different pixel intervals; wherein the period widths corresponding to the n Gray code images are respectively: N, 2N, ..., 2 n-1 N, or the cycle width is: 2 n-1 N, 2 n-2 N,…N.

7. The method for temporal encoding of space in three-dimensional reconstruction according to claim 4, characterized in that: The Gray coding of each pixel position in the space to be tested by using n Gray code images and their corresponding period widths includes: n+1+2 Gray code images are sequentially projected onto the measured space, wherein each projection of a Gray code image obtains a second captured image, and n+1+2 second captured images are obtained; the Gray code values ​​formed by the n+1+2 second captured images can identify at least M different pixel intervals; wherein the period widths corresponding to the n+1 Gray code images are respectively: 1 / 2N, N, 2N, ..., 2 n-1 N; or the cycle width is: 2 n-1 N, 2 n-2 N, ...N, 1 / 2N; the other two Gray code images are a full black Gray code image and a full white Gray code image respectively.

8. The method for temporal encoding of space in three-dimensional reconstruction according to claim 7, characterized in that: The Gray coding of each pixel position in the test space by using n Gray code images also includes: Determining a binarization threshold of the Gray code image based on the pixel grayscale values ​​of the second captured image corresponding to the all-black Gray code image and the all-white Gray code image; For any pixel position in the second captured image, if the pixel grayscale value corresponding to the pixel position is greater than the binarization threshold, the pixel grayscale value corresponding to the pixel position is set to 1; if the pixel grayscale value corresponding to the pixel position is less than or equal to the binarization threshold, the pixel grayscale value corresponding to the pixel position is set to 0.

9. The method for temporal encoding of space in three-dimensional reconstruction according to claim 1, characterized in that: The method of fitting a grayscale change curve for each pixel position of the space to be measured based on the change of the pixel grayscale value at the same pixel position in the N first captured images, and determining the time sequence value corresponding to the grayscale change peak value in the grayscale change curve as the folded time sequence value includes: For any pixel position, obtain N pixel grayscale values ​​corresponding to the pixel position in the N first captured images respectively; Fitting the N pixel grayscale values ​​into a grayscale change curve; Obtain the time interval [t1, t2] in which the grayscale value of the pixel in the grayscale change curve is greater than the grayscale threshold, and determine the time value corresponding to the grayscale change peak value of the pixel position in combination with the following formula; Among them, y is the time series value corresponding to the grayscale change peak value, [t1, t2] is the time series interval where the pixel grayscale value is greater than the grayscale threshold, I t is the pixel grayscale value corresponding to the time sequence t.

10. The method for temporal encoding of space in three-dimensional reconstruction according to claim 4, characterized in that: The unfolding of the folded timing value corresponding to each pixel position based on the Gray code values ​​corresponding to the M pixel intervals includes: For any pixel position, obtain the folded timing value and Gray code value corresponding to the pixel position, and determine the unfolded timing value corresponding to the pixel position by combining the following formula; T(i,j)=kN+t(i,j) Among them, k is the Gray code value corresponding to the pixel interval where the pixel position (i, j) is located, t(i, j) is the folded timing value corresponding to the pixel position (i, j), and T(i, j) is the unfolded timing value corresponding to the pixel position (i, j).

11. A method for calibrating the relationship between time sequence and height mapping in three-dimensional reconstruction, characterized in that: include: Step S100: placing a calibration object with known height data in a calibration plane of the space to be calibrated, wherein the height data is the height value of each pixel position in the space to be calibrated relative to the calibration plane; wherein a timing and height mapping relationship to be determined is preset for the space to be calibrated, and the relationship includes a plurality of unknown calibration polynomial coefficients; Step S200: using the time sequence coding method according to any one of claims 1 to 10 to obtain the unfolded time sequence value corresponding to each pixel position in the space to be calibrated; Step S300: determining whether the preset calibration measurement times P are met at present; if not, controlling the calibration plane to rise by a set distance, and continuing to execute step S200; if yes, executing step S400; Step S400: Substitute the height data and the expanded timing values ​​corresponding to the P groups of calibration measurements into the timing and height mapping relationship to determine the unknown calibration polynomial coefficients, thereby determining the timing and height mapping relationship.

12. The method for calibrating the time sequence and height mapping relationship in three-dimensional reconstruction according to claim 11, characterized in that: The relationship includes: h(i,j)=p w T(i,j) w +p w-1 T(i,j) w-1 +…+p1T(i,j)+p0 Among them, h(i,j) is the height data corresponding to the pixel position (i,j), p0 to p w is the calibration polynomial coefficient, w is the highest order of the polynomial, and T(i, j) is the expanded time value corresponding to the pixel position (i, j).

13. A three-dimensional reconstruction method, characterized in that: include: Using the time series coding method according to any one of claims 1 to 10, obtaining the expanded time series value corresponding to each pixel position in the measured space; Substituting the expanded timing value into a timing-height mapping relationship to obtain height data corresponding to each pixel position in the measured space; Based on the height data corresponding to each pixel position, the space to be measured is reconstructed in three dimensions.

14. The three-dimensional reconstruction method according to claim 13, characterized in that: The timing and height mapping relationship is: h(i,j)=p w T(i,j) w +p w-1 T(i,j) w-1 +…+p1T(i,j)+p0 Among them, h(i,j) is the height data corresponding to the pixel position (i,j), p0 to p w is the preset calibration polynomial coefficient, w is the highest order of the polynomial, and T(i,j) is the expanded time value corresponding to the pixel position (i,j).

15. A time series coding device, characterized in that: Used to execute the temporal encoding method of space in three-dimensional reconstruction as described in any one of claims 1 to 10.

16. A three-dimensional reconstruction system, characterized in that: The system comprises: A projection device for projecting a line-coded diagram; An imaging device, used for capturing a first captured image corresponding to the line coding image; A control device, used to execute the three-dimensional reconstruction method described in claim 13.

17. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the three-dimensional reconstruction method according to claim 13 is implemented.

18. An electronic device, comprising a storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the three-dimensional reconstruction method as claimed in claim 13 is implemented.

Images