Pattern generation method, system and apparatus for reconstruction of three-dimensional structures based on back-diffraction of structured light

Abstract

The application relates to a pattern generation method, system and device for structure light three-dimensional reconstruction based on back-diffraction, which comprises the following steps: S1, generating simulated stripes projected to a surface, recording as an image I1, the bit depth of the image I1 is b, the width and height of the image I1 are both A times of the DMD resolution, and A >= 2; S2, performing back-diffraction processing on the pattern according to a back-diffraction function selected according to an optical system; S3, performing once back convolution operation on the image I1 and the back-diffraction function to obtain an image I2; S4, performing down-sampling operation on the image I2 to obtain an image I3 consistent with the DMD resolution; S5, performing local binary processing on the image I3 to obtain an image I4 with a bit depth of 1 bit; and S6, replacing the original image I1 with the image I4. Through the process of simulating back-diffraction, the bit depth 1 bit image on the DMD is calculated, the 1 bit image achieves the same effect as the 8 bit image, the projection efficiency is improved by dozens of times, and thus the existing technical problems are solved.

Description

Technical Field

[0001] This invention relates to the field of computer vision inspection, and in particular to a pattern generation method, system, and apparatus based on structured light 3D reconstruction using back diffraction. Background Technology

[0002] Machine vision technology refers to using vision (i.e., the common method of taking pictures with a camera) to obtain information such as the color, texture and outline of an object, and then processing the obtained information with specific algorithms to achieve the goal of detection or measurement, thereby improving the quality monitoring of manufacturing, assembly and other processes.

[0003] Currently, technologies for measuring object contours, based on their principles, mainly include triangulation, interferometry, Time-of-Flight (TOF, used in lidar), and dispersion. Triangulation refers to the non-coaxial optical paths of the illumination and imaging systems, creating an angle that uniquely determines the three-dimensional coordinates of an object's points. Due to its high speed and accuracy, triangulation is widely used in the consumer electronics industry. Industrialized products based on triangulation include line-scan laser and structured light arrays. Line-scan lasers require external movement to obtain the complete contour of an object, making the process relatively cumbersome. Structured light arrays, on the other hand, can capture the object's three-dimensional information in a single image within the imaging field of view. Therefore, structured light technology is widely used in areas such as mobile phone screen inspection, robot navigation, and facial recognition, significantly improving production efficiency and user experience. Its high precision and rapid imaging advantages make it one of the key technologies in modern industry and intelligent devices.

[0004] Existing structured light technology typically uses spatial light modulators, most commonly the DLP (Digital Light Projector) modules manufactured by Texas Instruments (TI). DLP is essentially a projection system capable of projecting patterns with bit depths of 1 bit (also known as binary) and 8 bits, but typically the 1-bit projection frame rate is much higher than the 8-bit frame rate. For example, the DLP4500 module has a maximum 1-bit projection frame rate of 4225 Hz and an 8-bit pattern maximum projection frame rate of 120 Hz. A bit depth of 1 bit means that the projector has only two states: off and on, with no change in grayscale. A bit depth of 8 bits means that the projected brightness has… The change in level.

[0005] Structured light arrays typically require projecting multiple patterns with grayscale variations that follow a sinusoidal function to generate phase shift. In some specific applications described below, even higher requirements exist:

[0006] (1) The need for further improvement in frame rate. For example, it is necessary to test the shape change of a car airbag at the moment of explosion. Currently, high-speed cameras with a Hz of up to several thousand, such as 5000 Hz, are used to capture images. However, a camera alone cannot obtain the shape; it can only obtain two-dimensional information.

[0007] (2) The prerequisite for static shooting of structured light. In some high-precision semiconductor scenarios, such as the requirement to achieve a height measurement accuracy of micrometers (μm), there are vibrations caused by external excitations, which break the prerequisite for static shooting of structured light.

[0008] (3) Other scenarios that require high-speed area array structured light.

[0009] Whether the object being measured has a high speed or is due to vibration, the problem can essentially be solved by increasing the frame rate of the image. In other words, as long as the shooting speed is fast enough, the object is relatively stationary.

[0010] However, structured light arrays are based on the principle of phase shifting, which requires projecting a series of patterns with grayscale changes. Even when using a DLP4500 with an 8-bit image depth, only a frame rate of 120 Hz can be achieved, which cannot meet the needs of high-speed measurement. Summary of the Invention

[0011] The technical problem to be solved by this invention is to design a pattern generation method, system and device for structured light 3D reconstruction based on back diffraction. By simulating the back diffraction process, that is, if the pattern finally projected by the DLP is an image with sinusoidal grayscale change that meets the requirements of the area array structured light, then by calculating the bit depth 1-bit image on the DMD, the same effect as the 8-bit image can be achieved with a 1-bit image, thereby improving the projection efficiency by tens of times, thus solving the existing technical problem.

[0012] To address the aforementioned technical problems, this invention provides a pattern generation method based on structured light 3D reconstruction using anti-diffraction, specifically comprising the following steps:

[0013] Step S1: Generate simulated fringes projected onto the object surface and record them as image I1. The bit depth of image I1 is b, and the width and height of image I1 are both A times the resolution of the DMD, where A ≥ 2. Specifically, let the resolution of the DMD be W*H, where W represents the width of the image that can be loaded and H represents the height. Then the resolution of image I1 is AW*AH, where A ≥ 2.

[0014] Step S2: Select the inverse diffraction function according to the optical system to perform inverse diffraction processing on the pattern. Inverse diffraction processing includes calculations to improve image sharpness and restore the image.

[0015] Step S3: Perform a deconvolution operation on image I1 and the inverse diffraction function to obtain image I2.

[0016] Step S4: Downsample image I2 to obtain image I3 with the same resolution as the DMD.

[0017] Step S5: Perform local binarization on image I3 to obtain image I4 with a bit depth of 1 bit.

[0018] Step S6: Replace the original image I1 with image I4.

[0019] Furthermore, in step S1, image I1 is either vertical stripes or horizontal stripes. Vertical stripes refer to a periodic sinusoidal distribution of image grayscale along the horizontal direction, while horizontal stripes refer to a periodic sinusoidal distribution of image grayscale along the vertical direction.

[0020] Furthermore, in step S2, the inverse diffraction functions include Gaussian Model, Simulated Defocus Model, Born & Wolf Model, Gibson & Lanni Model, Variable Refractive Index Gibson & Lanni Model, and Richards & Wolf Model.

[0021] Furthermore, in step S5, the image binarization algorithm is either OTSU or Huang thresholding method or InterModes thresholding method or IsoData thresholding method or Li thresholding method or Max Entropy maximum entropy value segmentation or mean method segmentation or Min Error minimum error segmentation or Minimum minimum thresholding method or Moments geometric moment thresholding method or Percentile thresholding method or Renyi Entropy thresholding method or Shanbhag thresholding method or Yen thresholding method.

[0022] Furthermore, in step S5, the binarization process includes error diffusion: the error is diffused to the remaining pixels. The diffusion range or proportion satisfies the principle of energy conservation. Energy conservation means that the sum of the diffusion proportions is 100%, which is equal to the energy change of the target pixel in binarization. The diffusion proportion satisfies the following: the farther away from the target point, the lower the energy proportion; along the scanning direction, the lower the energy proportion.

[0023] Furthermore, the error propagation method involves spreading to the left, right, and next row, specifically including the following steps:

[0024] Step S511: Perform threshold segmentation on image I3. If the gray value of pixel I3(x,y) in image I3 is greater than the threshold T, then assign the value V of this pixel to 255; otherwise, assign the value to 0.

[0025] Step S512: Calculate the energy change , .

[0026] Step S513: Change the energy The diffusion extends to surrounding pixels, spreading outwards to pixels on the right, at a 45-degree angle to the lower left, directly below, and at a 45-degree angle to the lower right. , , and energy, It is a constant, and .

[0027] Step S514: Starting from the first row, perform a row scan from left to right until the last column is scanned.

[0028] Step S515: Starting from the row below the last column of pixels, perform a row scan from right to left, repeating steps S511 to S513, but adjusting the diffusion method in step S513 to diffuse to pixels in the left, lower left 45-degree, directly below, and lower right 45-degree directions respectively. , , and energy, It is a constant, and .

[0029] Step S516: Repeat steps S511 to S515 until the entire image is binarized to obtain the final image I4 with a bit depth of 1 bit.

[0030] Furthermore, error propagation can also occur by spreading to the left, right, and top rows, specifically including the following steps:

[0031] Step S521: Perform threshold segmentation on image I3. If the gray value of pixel I3(x,y) in image I3 is greater than the threshold T, then assign the value V of this pixel to 255; otherwise, assign the value 0.

[0032] Step S522: Calculate the energy of the change , .

[0033] Step S523: Change the energy It spreads to surrounding pixels, specifically to the right, at an upper left 45-degree angle, directly above, and at an upper right 45-degree angle. , , and energy, It is a constant, and .

[0034] Step S524: Starting from the last row, perform a row scan from left to right until the last column is scanned.

[0035] Step S525: Starting from the row above the last column of pixels, perform a row scan from right to left, repeating steps S521 to S523, but adjusting the diffusion method in step S523 to diffuse to pixels in the left, upper left 45-degree, top, and upper right 45-degree directions respectively. , , and energy, It is a constant, and .

[0036] Step S526: Repeat steps S521 to S525 until the entire image is binarized to obtain the final image I4 with a bit depth of 1 bit.

[0037] Furthermore, the range of the threshold T for segmenting image I3 is: .

[0038] This invention also provides a pattern generation system for structured light 3D reconstruction based on anti-diffraction, employing the aforementioned pattern generation method for structured light 3D reconstruction based on anti-diffraction, specifically including:

[0039] The pattern generation module is used to generate a simulated stripe image I1 projected onto the object surface.

[0040] The anti-diffraction processing module is used to select a suitable anti-diffraction function based on the optical system and perform anti-diffraction calculations on the pattern to obtain an image I2 that removes the influence of the optical system.

[0041] The downsampling module is used to downsample image I2 to obtain image I3 with the same resolution as the DMD.

[0042] The local binarization module is used to perform local binarization processing on image I3 to obtain image I4 with a bit depth of 1 bit.

[0043] The present invention also provides a pattern generation device for structured light three-dimensional reconstruction based on anti-diffraction, comprising:

[0044] At least one processor; and

[0045] At least one memory communicatively connected to the processor;

[0046] The memory stores instructions that can be executed by a processor, which are then executed by the processor to cause the device to perform the aforementioned pattern generation method for structured light 3D reconstruction based on anti-diffraction.

[0047] The beneficial effects of this invention are:

[0048] (1) This invention can replace the common 8-bit stripes with calculated 1-bit stripes when projecting structured light stripes. Furthermore, the 1-bit projection frame rate of commonly used DLP projection systems, regardless of model, is far higher than that of 8-bit images. This greatly improves the scanning speed of 3D scanning devices based on DLP projection systems.

[0049] (2) The present invention greatly improves the scanning speed and can be applied to scenarios that require high-speed imaging, such as providing a solution for the state detection of airbags at the moment of explosion.

[0050] (3) This invention provides a low-cost solution for applications requiring high-precision measurement (such as the measurement of semiconductor microstructures) in addition to adding active vibration isolation. The cost of active vibration isolation is several times that of ordinary vibration isolation (such as damping vibration isolation). Attached Figure Description

[0051] The specific embodiments of the present invention will be further explained below with reference to the accompanying drawings.

[0052] Figure 1 This is a flowchart of the pattern generation method for structured light 3D reconstruction based on reverse diffraction according to the present invention.

[0053] Figure 2 This is a schematic diagram of an immersion microscope system.

[0054] Figure 3 This is a system block diagram of the pattern generation system for structured light 3D reconstruction based on anti-diffraction according to the present invention. Detailed Implementation

[0055] The pattern generation method for structured light 3D reconstruction based on anti-diffraction of the present invention specifically includes the following steps:

[0056] Step S1: Generate simulated fringes projected onto the object surface and record them as image I1. The bit depth of image I1 is b, and the width and height of image I1 are both A times the DMD resolution, and A≥2.

[0057] Furthermore, in step S1, image I1 is either vertical stripes or horizontal stripes. Vertical stripes refer to a periodic sinusoidal distribution of image grayscale along the horizontal direction, while horizontal stripes refer to a periodic sinusoidal distribution of image grayscale along the vertical direction.

[0058] Specifically, in this step, image I1 is a sinusoidal fringe, and the expression for image I1 is:

[0059] ;

[0060] in, Indicates the background grayscale of the pattern. Indicates adjustment system, Indicates the period of the stripes, Representing the pixels of an image Coordinates, i.e. along When the grayscale value changes in direction (also referring to the horizontal direction), it produces periodic vertical stripes.

[0061] If the resolution of the DMD is W*H, where W represents the width of the image that can be loaded and H represents the height, then according to the Nyquist theorem, the sampling frequency must be greater than or equal to twice the maximum frequency to represent a signal. Therefore, the resolution of image I1 must be AW*AH, where A≥2, meaning that a region corresponds to a pixel on the DMD.

[0062] Step S2: Select the inverse diffraction function according to the optical system to perform inverse diffraction processing on the pattern. Inverse diffraction processing includes calculations to improve image sharpness and restore the image.

[0063] Furthermore, in step S2, the de-diffraction function (DDF) includes the Gaussian Model (DDF-GM), the Simulated Defocus Model (DDF-SDM), the Born & Wolf Model (DDF-BWM), the Gibson & Lanni Model (DDF-GLM), the Variable Refractive Index Gibson & Lanni Model (DDF-VRIGLM), and the Richards & Wolf Model (DDF-RWM).

[0064] Specifically, in this step, the DDF-GM model can be used for all optical imaging systems, and its expression is:

[0065] ;

[0066] in Represents coordinates in three-dimensional space. Indicates the direction along the optical axis of the optical system. and The plane orthogonal to the optical axis. Parameters This is the variance in the Gaussian equation. The larger the value, the more dispersed the Gaussian distribution, and the blurrier the image. As can be seen from the above formula, the variance... It is the depth of the optical axis The relevant quantities. As can be seen, the entire DDF-GM model describes how the further away from the focal plane, the blurrier the image.

[0067] If we consider the Fourier frequency of the DDF-GM model, it becomes the DDF-SDM.

[0068] Clearly, both the Gaussian-based spatial processing model DDF-GM and the frequency-based model DDF-SDM only consider geometric factors, i.e. The impact of direction was not taken into account:

[0069] 1. Due to the influence of the optical system's magnification and the optical numerical aperture (NA).

[0070] 2. The refractive index of the medium, sample, or glass cover.

[0071] 3. Different wavelengths, or wavenumbers.

[0072] 4. Phase aberrations, etc.

[0073] Therefore, for implementers who do not require very precise implementation, the DDF-GM or DDF-SDM model can be used directly.

[0074] For specialized imaging systems, such as immersion microscope systems,

[0075] Combination Figure 2 As shown, the objective lens is the microscope objective, the immersion layer is the immersion liquid (usually called the medium), typically water, which is used to increase the refractive index, thereby achieving a larger light-gathering angle and higher resolution. The cover slip is a glass cover plate. The mounting slide is the stage on which the sample layer is placed. A more complete model is the DDF-GLM model:

[0076] ;

[0077] ;

[0078] in, For geometric information, Numerical aperture, The refractive index of the sample, The axial position in the sample. and The true and nominal refractive indices of the glass cover (usually equal). and These are the true refractive index and the nominal refractive index of the medium (usually equal). and This refers to the actual thickness and nominal thickness of the glass cover (which are usually equal). and The distance from the actual and nominal coverslip to the objective lens (usually equal). For wave number, This is the axial distance between the detector and the telescope, usually the focal length of the telescope. The expression for phase aberration is given by [expression]. Let C be the zeroth-order component of the Bessel algorithm, and C be the normalization constant. For magnification, It is a quantity used for integration, from As can be seen from the calculation formula, it performs an integral operation on the numerical aperture NA from 0 to 1, that is, it integrates over different light-receiving ranges.

[0079] In the DDF-GLM model, the refractive index of the sample is assumed to be constant. The DDF-VRIGLM model is an extension of the DDF-GLM model, which additionally considers the phase aberration caused by the refractive index variation within the sample.

[0080] Step S3: Perform a deconvolution operation on image I1 and the inverse diffraction function to obtain image I2 with the influence of the optical system removed.

[0081] Step S4: Downsample image I2 to obtain image I3 with the same resolution as the DMD. Image I3 and the DMD require the same W*H size.

[0082] Step S5: Using the principle of local dot density and energy conservation, perform local binarization on image I3, that is, the higher the gray level, the more bright spots there are, and the lower the gray level, the more dark spots there are. Scan line by line in a back-and-forth pattern to obtain image I4 with a bit depth of 1 bit.

[0083] Furthermore, in step S5, the image binarization algorithm is either OTSU or Huang thresholding method or InterModes thresholding method or IsoData thresholding method or Li thresholding method or Max Entropy maximum entropy value segmentation or mean method segmentation or Min Error minimum error segmentation or Minimum minimum thresholding method or Moments geometric moment thresholding method or Percentile thresholding method or Renyi Entropy thresholding method or Shanbhag thresholding method or Yen thresholding method.

[0084] For scenarios requiring strict energy conservation, where the application is sensitive to light intensity fluctuations, the Min Error segmentation method is recommended. This method iterates through the segmentation thresholds to find the image before and after binarization that best approximates energy conservation. If a higher tolerance for error is desired, the mean method can be used directly. This method averages the grayscale values ​​within a local area, such as a 100x100 pixel region, as the segmentation threshold. However, this method introduces a significant error in the energy values ​​of that region before and after segmentation, and does not strictly satisfy energy conservation.

[0085] Furthermore, in step S5, the binarization process includes error diffusion: the error is diffused to the remaining pixels. The diffusion range or proportion satisfies the principle of energy conservation. Energy conservation means that the sum of the diffusion proportions is 100%, which is equal to the energy change of the target pixel in binarization. The diffusion proportion satisfies the following: the farther away from the target point, the lower the energy proportion; along the scanning direction, the lower the energy proportion.

[0086] Furthermore, the error propagation method involves spreading to the left, right, and next row, specifically including the following steps:

[0087] Step S511: Perform threshold segmentation on image I3. If the gray value of pixel I3(x,y) in image I3 is greater than the threshold T, then assign the value V of this pixel to 255; otherwise, assign the value to 0.

[0088] Step S512: Calculate the energy change , .

[0089] Step S513: Change the energy The diffusion extends to surrounding pixels, spreading outwards to pixels on the right, at a 45-degree angle to the lower left, directly below, and at a 45-degree angle to the lower right. , , and energy, It is a constant, and .

[0090] Step S514: Starting from the first row, perform a row scan from left to right until the last column is scanned.

[0091] Step S515: Starting from the row below the last column of pixels, perform a row scan from right to left, repeating steps S511 to S513, but adjusting the diffusion method in step S513 to diffuse to pixels in the left, lower left 45-degree, directly below, and lower right 45-degree directions respectively. , , and energy, It is a constant, and .

[0092] Step S516: Repeat steps S511 to S515 until the entire image is binarized to obtain the final image I4 with a bit depth of 1 bit.

[0093] Furthermore, error propagation can also occur by spreading to the left, right, and top rows, specifically including the following steps:

[0094] Step S521: Perform threshold segmentation on image I3. If the gray value of pixel I3(x,y) in image I3 is greater than the threshold T, then assign the value V of this pixel to 255; otherwise, assign the value 0.

[0095] Step S522: Calculate the energy of the change , .

[0096] Step S523: Change the energy It spreads to surrounding pixels, specifically to the right, at an upper left 45-degree angle, directly above, and at an upper right 45-degree angle. , , and energy, It is a constant, and .

[0097] Step S524: Starting from the last row, perform a row scan from left to right until the last column is scanned.

[0098] Step S525: Starting from the row above the last column of pixels, perform a row scan from right to left, repeating steps S521 to S523, but adjusting the diffusion method in step S523 to diffuse to pixels in the left, upper left 45-degree, top, and upper right 45-degree directions respectively. , , and energy, It is a constant, and .

[0099] Step S526: Repeat steps S521 to S525 until the entire image is binarized to obtain the final image I4 with a bit depth of 1 bit.

[0100] Furthermore, the range of the threshold T for segmenting image I3 is: .

[0101] Step S6: Replace the original image I1 with image I4, so that all images loaded on the DMD are 1-bit images, which greatly improves the projection frame rate.

[0102] Example 1

[0103] In this embodiment, the DLP4710 is used, with a DMD resolution of 1920*1080 and a bit depth of 8 bits. The optical system used with it is a dual telecentric system, and the medium between it and the observed object is air.

[0104] Combination Figure 1 The pattern generation method for structured light 3D reconstruction based on anti-diffraction in this embodiment specifically includes the following steps:

[0105] Step S1: Generate normal stripes based on an 8-bit bit depth. The formula for representing the stripes is as follows:

[0106] ;

[0107] Based on the above formula, we obtain a stripe pattern with a minimum gray level of 0 and a maximum gray level of 255. The stripe period is 100 pixels and runs along the edge of the image. Change the grayscale along the direction. The directional grayscale value remains unchanged.

[0108] To ensure adequate sampling, according to the Nyquist theorem, the signal must be sampled at twice the original frequency. The higher the multiplier, the higher the sampling frequency, and the closer the signal will be to the original analog signal. This embodiment uses a 3x sampling ratio, meaning the width and height of the generated image are both three times that of the DMD. The resolution is (1920*3) × (1080*3) = 5760 × 3240. It should be noted that the sampling multiple A only needs to satisfy A≥2. This embodiment only takes 3 times sampling as an example. Sampling multiples of other multiples all fall within the protection scope of this application.

[0109] Step S2: Since all the fringe images are only images loaded into the DMD system and do not affect the subsequent optical system, all images can be processed using the same inverse diffraction function. In this embodiment, the optical system is limited to a non-immersion type, i.e., the medium is air. Therefore, this embodiment selects the simplest Gaussian model, i.e., the DDF-GM model, for inverse diffraction processing.

[0110] Step S3: Perform a deconvolution operation on image I1 and the inverse diffraction function to obtain image I2.

[0111] Step S4: Use the mean method to downsample image I2 to obtain image I3 with the same resolution as DMD; then the gray level of the pixels in image I3 is the average value of the 3*3 area in image I2.

[0112] Step S5: Using the principle of local lattice density and energy conservation, perform local binarization on image I3 and scan line by line in a back-shaped pattern to obtain image I4 with a bit depth of 1 bit.

[0113] In this embodiment, the binarization process specifically includes error diffusion: the error is diffused to the remaining pixels. The diffusion range or proportion satisfies the principle of energy conservation. Energy conservation means that the sum of the diffusion proportions is 100%, which is equal to the energy change of the target pixel in binarization. The diffusion proportion satisfies the following: the farther away from the target point, the lower the energy proportion; along the scanning direction, the lower the energy proportion.

[0114] In this preferred embodiment, the error propagation method is to propagate to the left, right, and next row, specifically including the following steps:

[0115] Step S511: Perform threshold segmentation on image I3. If the gray value of pixel I3(x,y) in image I3 is greater than the threshold T, then assign the value V of this pixel to 255; otherwise, assign the value to 0.

[0116] In this preferred embodiment, the range of the threshold T is: , Given the bit depth of image I1, in this embodiment, the bit depth of image I1 is 8 bits, and the threshold T can be 127. Then, for image I3, if the gray value of pixel I3(x,y) is greater than 127, this pixel value is assigned to 255; otherwise, it is assigned to 0. That is, the pixel value V has only two values: 0 and 255.

[0117] Step S512: Calculate the energy change , , It refers to the change in pixel value before and after the assignment.

[0118] Step S513: Change the energy The diffusion extends to surrounding pixels, spreading outwards to pixels on the right, at a 45-degree angle to the lower left, directly below, and at a 45-degree angle to the lower right. , , and energy, It is a constant, and .

[0119] In this embodiment, the diffusion ratio is specifically set as follows: , , , It should be noted that the diffusion ratio is not a fixed value. The basic principle is that the farther away from the target pixel value, the lower the diffusion energy; at the same distance, the diffusion energy along the scanning direction is lower.

[0120] Step S514: Starting from the first row, perform a row scan from left to right until the last column is scanned.

[0121] Step S515: Starting from the row below the last column of pixels, perform a row scan from right to left, repeating steps S511 to S513, but adjusting the diffusion method in step S513 to diffuse to pixels in the left, lower left 45-degree, directly below, and lower right 45-degree directions respectively. , , and energy, It is a constant, and .

[0122] In this embodiment, the diffusion ratio is specifically set as follows: , , , .

[0123] Step S516: Repeat steps S511 to S515 until the entire image is binarized to obtain the final image I4 with a bit depth of 1 bit.

[0124] Step S6: Replace all 8-bit phase-shifted fringe images that need to be projected with the image I4 that has been locally binarized in step S5 before projection.

[0125] After this step is completed, all images to be projected are 1 bit. Taking the DLP4500 as an example, the time required to project one image is originally 8.3 ms at a frame rate of 120 Hz. Using the method in this embodiment, it can be reduced to 0.24 ms at a frame rate of 4225 Hz, which is 35 times faster.

[0126] Example 2

[0127] The difference between the pattern generation method of structured light 3D reconstruction based on anti-diffraction in this embodiment and that in embodiment 1 is the different error diffusion method when performing local binarization processing on image I3 in step S5. The other technical features are the same as those in embodiment 1.

[0128] In this embodiment, the error propagation method is to propagate to the left, right, and top rows, specifically including the following steps:

[0129] Step S521: Perform threshold segmentation on image I3. If the gray value of pixel I3(x,y) in image I3 is greater than the threshold T, then assign the value V of this pixel to 255; otherwise, assign the value 0.

[0130] In this preferred embodiment, the range of the threshold T is: , Given the bit depth of image I1, in this embodiment, the bit depth of image I1 is 8 bits, and the threshold T can be 132. Then, for image I3, if the gray value of pixel I3(x,y) is greater than 132, this pixel value is assigned to 255; otherwise, it is assigned to 0. That is, the pixel value V has only two values: 0 and 255.

[0131] Step S522: Calculate the energy of the change , , It refers to the change in pixel value before and after the assignment.

[0132] Step S523: Change the energy It spreads to surrounding pixels, specifically to the right, at an upper left 45-degree angle, directly above, and at an upper right 45-degree angle. , , and energy, It is a constant, and .

[0133] In this embodiment, the diffusion ratio is specifically set as follows: , , , It should be noted that the diffusion ratio is not a fixed value. The basic principle is that the farther away from the target pixel value, the lower the diffusion energy; at the same distance, the diffusion energy along the scanning direction is lower.

[0134] Step S524: Starting from the last row, perform a row scan from left to right until the last column is scanned.

[0135] Step S525: Starting from the row above the last column of pixels, perform a row scan from right to left, repeating steps S521 to S523, but adjusting the diffusion method in step S523 to diffuse to pixels in the left, upper left 45-degree, top, and upper right 45-degree directions respectively. , , and energy, It is a constant, and ;

[0136] In this embodiment, the diffusion ratio is specifically set as follows: , , , .

[0137] Step S526: Repeat steps S521 to S525 until the entire image is binarized to obtain the final image I4 with a bit depth of 1 bit.

[0138] Example 3

[0139] In this embodiment, the DLP4710 is used, with a DMD resolution of 1920*1080 and a bit depth of 8 bits. The optical system used with it is a dual telecentric system, and the medium between it and the observed object is air.

[0140] Combination Figure 1 The pattern generation method for structured light 3D reconstruction based on anti-diffraction in this embodiment specifically includes the following steps:

[0141] Step S1: Generate normal stripes based on an 8-bit bit depth. The formula for representing the stripes is as follows:

[0142] ;

[0143] Based on the above formula, we obtain a stripe pattern with a minimum gray level of 0 and a maximum gray level of 255. The stripe period is 100 pixels and runs along the edge of the image. Change the grayscale in the direction (i.e., along the vertical direction), along The directional grayscale value remains unchanged, generating periodic horizontal stripes.

[0144] To ensure adequate sampling, according to the Nyquist theorem, the signal must be sampled at twice the original frequency. The higher the multiplier, the higher the sampling frequency, and the closer the signal will be to the original analog signal. This embodiment uses a 3x sampling ratio, meaning the width and height of the generated image are both three times that of the DMD. The resolution is (1920*3) × (1080*3) = 5760 × 3240. It should be noted that the sampling multiple A only needs to satisfy A≥2. This embodiment only takes 3 times sampling as an example. Sampling multiples of other multiples all fall within the protection scope of this application.

[0145] The remaining steps S2 to S6 are the same as those in Example 1 or Example 2.

[0146] Example 4

[0147] In this embodiment, the DLP4710 is used, with a DMD resolution of 1920*1080 and a bit depth of 8 bits. The optical system used with it is a pinhole system, and the medium between it and the observed object is air.

[0148] Combination Figure 1 The pattern generation method based on structured light 3D reconstruction using anti-diffraction in this embodiment is the same as the method in Embodiment 1, Embodiment 2, or Embodiment 3.

[0149] Example 5

[0150] In this embodiment, the DLP4500 is used, with a DMD resolution of 912*1140 and a bit depth of 8 bits. The optical system used with it is a pinhole system, and the medium between it and the observed object is air.

[0151] Combination Figure 1 The pattern generation method for structured light 3D reconstruction based on anti-diffraction in this embodiment specifically includes the following steps:

[0152] Step S1: Generate normal stripes based on an 8-bit bit depth. The formula for representing the stripes is as follows:

[0153] ;

[0154] Based on the above formula, we obtain a stripe pattern with a minimum gray level of 0 and a maximum gray level of 255. The stripe period is 100 pixels and runs along the edge of the image. Change the grayscale along the direction. The directional grayscale value remains unchanged.

[0155] To ensure adequate sampling, according to the Nyquist theorem, the signal must be sampled at twice the original frequency. The higher the multiplier, the higher the sampling frequency, and the closer the signal will be to the original analog signal. This embodiment uses a 3x sampling ratio, meaning the width and height of the generated image are both three times that of the DMD. The resolution is (912*3) × (1140*3) = 2736 × 3420. It should be noted that the sampling multiple A only needs to satisfy A≥2. This embodiment only takes 3 times sampling as an example. Sampling multiples of other multiples all fall within the protection scope of this application.

[0156] The remaining steps S2 to S6 are the same as those in Example 1, Example 2, Example 3, or Example 4.

[0157] Example 6

[0158] In this embodiment, the DLP4500 is used, with a DMD resolution of 912*1140 and a bit depth of 8 bits. The optical system used with it is a dual telecentric system, and the medium between it and the observed object is air.

[0159] The pattern generation method for structured light 3D reconstruction based on anti-diffraction in this embodiment is the same as the method in Embodiment 5.

[0160] Example 7

[0161] In this embodiment, the DLP4710 is used, with a DMD resolution of 1920*1080 and a bit depth of 8 bits. The optical system used with it is a dual telecentric system, and the medium between it and the observed object is water.

[0162] Combination Figure 1 The technical feature that distinguishes the pattern generation method of structured light three-dimensional reconstruction based on anti-diffraction in this embodiment from the methods in Embodiment 1, Embodiment 2, or Embodiment 3 is the different anti-diffraction function selected in step S2. The remaining steps are the same as those in Embodiment 1, Embodiment 2, or Embodiment 3.

[0163] In this embodiment, specifically, in step S2, DDF-GLM, DDF-VARGLM, DDF-BWM, or DDF-RWM are selected as the inverse diffraction function for calculation.

[0164] Example 8

[0165] In this embodiment, the DLP4710 is used, with a DMD resolution of 1920*1080 and a bit depth of 8 bits. The optical system used with it is a pinhole system, and the medium between the DLP4710 and the observed object is water.

[0166] Combination Figure 1 The pattern generation method based on structured light 3D reconstruction using anti-diffraction in this embodiment is the same as the method in Embodiment 7.

[0167] Example 9

[0168] In this embodiment, a DLP4500 is used, with a DMD resolution of 912*1140 and a bit depth of 8 bits. The optical system used with it is a dual telecentric system, and the medium between the DLP4500 and the observed object is water.

[0169] Combination Figure 1 The technical feature that distinguishes the pattern generation method of structured light three-dimensional reconstruction based on anti-diffraction in this embodiment from the methods in Embodiment 1, Embodiment 2, or Embodiment 3 is the different anti-diffraction function selected in step S2. The remaining steps are the same as those in Embodiment 1, Embodiment 2, or Embodiment 3.

[0170] In this embodiment, specifically, in step S2, DDF-GLM, DDF-VARGLM, DDF-BWM, DDF-BWM, or DDF-RWM are selected as the inverse diffraction function for calculation.

[0171] Example 10

[0172] In this embodiment, a DLP4500 is used, with a DMD resolution of 912*1140 and a bit depth of 8 bits. The optical system used with it is a pinhole system, and the medium between the DLP4500 and the observed object is water.

[0173] Combination Figure 1 The pattern generation method based on structured light 3D reconstruction using anti-diffraction in this embodiment is the same as the method in Embodiment 9.

[0174] Example 11

[0175] Combination Figure 3 The pattern generation system for structured light 3D reconstruction based on anti-diffraction in this embodiment adopts any one of the pattern generation methods for structured light 3D reconstruction based on anti-diffraction from Embodiments 1 to 10, specifically including:

[0176] The pattern generation module is used to generate a simulated stripe image I1 projected onto the object surface.

[0177] The anti-diffraction processing module is used to select a suitable anti-diffraction function based on the optical system and perform anti-diffraction calculations on the pattern to obtain an image I2 that removes the influence of the optical system.

[0178] The downsampling module is used to downsample image I2 to obtain image I3 with the same resolution as the DMD.

[0179] The local binarization module is used to perform local binarization processing on image I3 to obtain image I4 with a bit depth of 1 bit.

[0180] Example 12

[0181] The pattern generation device for structured light 3D reconstruction based on anti-diffraction in this embodiment includes:

[0182] At least one processor; and

[0183] At least one memory communicatively connected to the processor;

[0184] The memory stores instructions that can be executed by a processor, which are executed by the processor to cause the device to perform any of the pattern generation methods for structured light three-dimensional reconstruction based on anti-diffraction in Embodiments 1 to 10.

[0185] Many specific details have been set forth in the foregoing description to provide a thorough understanding of the present invention. However, the above description is merely a preferred embodiment of the present invention, and the present invention can be implemented in many other ways different from those described herein. Therefore, the present invention is not limited to the specific embodiments disclosed above. Furthermore, any person skilled in the art can make many possible variations and modifications to the technical solutions of the present invention, or modify them into equivalent embodiments, using the methods and techniques disclosed above, without departing from the scope of the present invention. Any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention, without departing from the content of the present invention, shall still fall within the protection scope of the present invention.

Claims

1. A pattern generation method for three-dimensional reconstruction of structured light based on back diffraction, characterized in that: Includes the following steps: Step S1: Generate simulated fringes projected onto the object surface and record them as image I1. The bit depth of image I1 is b, and the width and height of image I1 are both A times the DMD resolution, and A≥2. Step S2: Select the anti-diffraction function according to the optical system to perform anti-diffraction processing on the pattern. The anti-diffraction processing includes calculations to improve image sharpness and restore the image. Step S3: Perform a deconvolution operation on image I1 and the inverse diffraction function to obtain image I2; Step S4: Downsample image I2 to obtain image I3 with the same resolution as the DMD; Step S5: Perform local binarization on image I3 to obtain image I4 with a bit depth of 1 bit; Step S6: Replace the original image I1 with image I4 and project it to obtain an image loaded on the DMD where all images are 1-bit. In step S5, the binarization process includes error diffusion: the error is diffused to the remaining pixels. The diffusion range or proportion satisfies the principle of energy conservation. Energy conservation means that the sum of the diffusion proportions is 100%, which is equal to the energy change of the target pixel in binarization. The diffusion proportions satisfy the following: the farther away from the target point, the lower the energy proportion; along the scanning direction, the lower the energy proportion. The error propagation method involves spreading to the left, right, and next row, specifically including the following steps: Step S511: Perform threshold segmentation on image I3. If the gray value of pixel I3(x,y) in image I3 is greater than the threshold T, then assign the value V of this pixel to 255; otherwise, assign the value to 0. Let I be the pixel coordinates of image I3, and V be the value assigned to the pixel after thresholding. Step S512: Calculate the energy change , ; Step S513: Change the energy The diffusion extends to surrounding pixels, spreading outwards to pixels on the right, at a 45-degree angle to the lower left, directly below, and at a 45-degree angle to the lower right. , , and energy, It is a constant, and ; Step S514: Starting from the first row, perform a row scan from left to right until the last column is scanned; Step S515: Starting from the row below the last column of pixels, perform a row scan from right to left, repeating steps S511 to S513, but adjusting the diffusion method in step S513 to diffuse to pixels in the left, lower left 45-degree, directly below, and lower right 45-degree directions respectively. , , and energy, It is a constant, and ; Step S516: Repeat steps S511 to S515 until the entire image is binarized to obtain the final image I4 with a bit depth of 1 bit.

2. The pattern generation method for structured light 3D reconstruction based on anti-diffraction according to claim 1, characterized in that: In step S1, image I1 is either vertical or horizontal stripes. Vertical stripes refer to a periodic sinusoidal distribution of image grayscale along the horizontal direction, while horizontal stripes refer to a periodic sinusoidal distribution of image grayscale along the vertical direction.

3. The pattern generation method for structured light 3D reconstruction based on anti-diffraction according to claim 1, characterized in that: In step S2, the inverse diffraction functions include Gaussian Model, Simulated Defocus Model, Born & Wolf Model, Gibson & Lanni Model, Variable Refractive Index Gibson & Lanni Model, and Richards & Wolf Model.

4. The pattern generation method for structured light three-dimensional reconstruction based on anti-diffraction according to claim 1, characterized in that: In step S5, the image binarization algorithm is either OTSU or Huang thresholding, or InterModes thresholding, or IsoData thresholding, or Li thresholding, or Max Entropy thresholding, or mean-based segmentation, or Min Error thresholding, or Minimum thresholding, or Moments thresholding, or Percentile thresholding, or Renyi Entropy thresholding, or Shanbhag thresholding, or Yen thresholding.

5. The pattern generation method for structured light three-dimensional reconstruction based on anti-diffraction according to claim 1, characterized in that: The range of the threshold T for segmenting image I3 is: .

6. A pattern generation system for structured light 3D reconstruction based on anti-diffraction, characterized in that: The pattern generation method based on structured light 3D reconstruction according to any one of claims 1-5 specifically includes: The pattern generation module is used to generate a simulated stripe image I1 projected onto the object surface; The anti-diffraction processing module is used to select an appropriate anti-diffraction function based on the optical system and perform anti-diffraction calculations on the pattern to obtain an image I2 that removes the influence of the optical system. The downsampling module is used to downsample image I2 to obtain image I3 with the same resolution as the DMD; The local binarization module is used to perform local binarization processing on image I3 to obtain image I4 with a bit depth of 1 bit.

7. A pattern generation device for structured light three-dimensional reconstruction based on anti-diffraction, characterized in that: include: At least one processor; as well as At least one memory communicatively connected to the processor; The memory stores instructions that can be executed by a processor, which are executed by the processor to cause the device to perform the pattern generation method for structured light three-dimensional reconstruction based on anti-diffraction as described in any one of claims 1-5.

8. A pattern generation method for three-dimensional reconstruction of structured light based on back diffraction, characterized in that: Includes the following steps: Step S1: Generate simulated fringes projected onto the object surface and record them as image I1. The bit depth of image I1 is b, and the width and height of image I1 are both A times the DMD resolution, and A≥2. Step S2: Select the anti-diffraction function according to the optical system to perform anti-diffraction processing on the pattern. The anti-diffraction processing includes calculations to improve image sharpness and restore the image. Step S3: Perform a deconvolution operation on image I1 and the inverse diffraction function to obtain image I2; Step S4: Downsample image I2 to obtain image I3 with the same resolution as the DMD; Step S5: Perform local binarization on image I3 to obtain image I4 with a bit depth of 1 bit; Step S6: Replace the original image I1 with image I4 and project it to obtain an image loaded on the DMD where all images are 1-bit. In step S5, the binarization process includes error diffusion: the error is diffused to the remaining pixels. The diffusion range or proportion satisfies the principle of energy conservation. Energy conservation means that the sum of the diffusion proportions is 100%, which is equal to the energy change of the target pixel in binarization. The diffusion proportions satisfy the following: the farther away from the target point, the lower the energy proportion; along the scanning direction, the lower the energy proportion. The error propagation method involves spreading to the left, right, and top rows, specifically including the following steps: Step S521: Perform threshold segmentation on image I3. If the gray value of pixel I3(x,y) in image I3 is greater than the threshold T, then assign the value V of this pixel to 255; otherwise, assign the value to 0. Let I be the pixel coordinates of image I3, and V be the value assigned to the pixel after thresholding. Step S522: Calculate the energy of the change , ; Step S523: Change the energy It spreads to surrounding pixels, specifically to the right, at an upper left 45-degree angle, directly above, and at an upper right 45-degree angle. , , and energy, It is a constant, and ; Step S524: Starting from the last row, perform a row scan from left to right until the last column is scanned; Step S525: Starting from the row above the last column of pixels, perform a row scan from right to left, repeating steps S521 to S523, but adjusting the diffusion method in step S523 to diffuse to pixels in the left, upper left 45-degree, top, and upper right 45-degree directions respectively. , , and energy, It is a constant, and ; Step S526: Repeat steps S521 to S525 until the entire image is binarized to obtain the final image I4 with a bit depth of 1 bit.

9. The pattern generation method for structured light three-dimensional reconstruction based on anti-diffraction according to claim 8, characterized in that: In step S1, image I1 is either vertical or horizontal stripes. Vertical stripes refer to a periodic sinusoidal distribution of image grayscale along the horizontal direction, while horizontal stripes refer to a periodic sinusoidal distribution of image grayscale along the vertical direction.

10. The pattern generation method for structured light three-dimensional reconstruction based on anti-diffraction according to claim 8, characterized in that: In step S2, the inverse diffraction functions include Gaussian Model, Simulated Defocus Model, Born & Wolf Model, Gibson & Lanni Model, Variable Refractive Index Gibson & Lanni Model, and Richards & Wolf Model.

11. The pattern generation method for structured light three-dimensional reconstruction based on anti-diffraction according to claim 8, characterized in that: In step S5, the image binarization algorithm is either OTSU or Huang thresholding, or InterModes thresholding, or IsoData thresholding, or Li thresholding, or Max Entropy thresholding, or mean-based segmentation, or MinError thresholding, or Minimum thresholding, or Moments thresholding, or Percentile thresholding, or Renyi Entropy thresholding, or Shanbhag thresholding, or Yen thresholding.

12. The pattern generation method for structured light three-dimensional reconstruction based on anti-diffraction according to claim 8, characterized in that: The range of the threshold T for segmenting image I3 is: .

13. A pattern generation system for structured light 3D reconstruction based on anti-diffraction, characterized in that: The pattern generation method based on structured light 3D reconstruction according to any one of claims 8-12 specifically includes: The pattern generation module is used to generate a simulated stripe image I1 projected onto the object surface; The anti-diffraction processing module is used to select an appropriate anti-diffraction function based on the optical system and perform anti-diffraction calculations on the pattern to obtain an image I2 that removes the influence of the optical system. The downsampling module is used to downsample image I2 to obtain image I3 with the same resolution as the DMD; The local binarization module is used to perform local binarization processing on image I3 to obtain image I4 with a bit depth of 1 bit.

14. A pattern generation device for structured light three-dimensional reconstruction based on anti-diffraction, characterized in that: include: At least one processor; as well as At least one memory communicatively connected to the processor; The memory stores instructions that can be executed by a processor, which are executed by the processor to cause the device to perform the pattern generation method for structured light three-dimensional reconstruction based on anti-diffraction as described in any one of claims 8-12.

Images