A three-dimensional imaging device and its usage method

Abstract

This invention discloses a three-dimensional imaging device and its usage method, including an optical system, a sensor module, a stepping control module, and an image processing unit. The optical system is constructed with a stepable sensor module positioned behind it. The photosensitive chip of the sensor module and the stepping control module are connected to a host computer. The host computer can control the stepping control module to move forward and backward, issue image acquisition commands, and acquire image data from the photosensitive chip of the sensor module. During system operation, light enters the optical system from the object side and reaches the image side. The host computer controls the sensor module to step, and the photosensitive chip acquires image data with each step. After acquiring multiple frames of image data, light field data can be obtained, and the three-dimensional model of the image side can be reconstructed. Because the imaging system is object-image conjugate, external three-dimensional information can also be obtained.

Description

Technical Field

[0001] This invention relates to the field of three-dimensional imaging technology, specifically a three-dimensional imaging device and its usage method. Background Technology

[0002] Currently, 3D imaging technology has wide applications in fields such as autonomous driving, robot navigation, and industrial inspection. Mainstream 3D imaging schemes include Time-of-Flight (TOF), structured light, and binocular vision. TOF calculates distance by measuring the flight time of light pulses, but suffers from low edge accuracy, requires high-frequency, high-intensity pulse emission, and has high device requirements. Structured light reconstructs 3D shapes by projecting patterns and analyzing deformation, but performs poorly in outdoor environments, is susceptible to ambient light interference, is only applicable to static scenes, and has limited measurement distance. Binocular vision, based on stereo matching principles, calculates depth through the parallax of two cameras, but involves high computational load, is sensitive to ambient lighting, and its matching accuracy is significantly affected by texture. These methods struggle to achieve high-precision, real-time 3D imaging, especially performing poorly in dynamic scenes. Therefore, this invention provides a 3D imaging device and its usage method. Summary of the Invention

[0003] To address the shortcomings of existing technologies, this invention provides a three-dimensional imaging device and its usage method, aiming to solve the aforementioned problems.

[0004] To achieve the above objectives, the present invention provides the following technical solution: a three-dimensional imaging device, comprising:

[0005] An optical system for receiving an object-side light field and mapping it onto an image-side light field;

[0006] The sensor module includes at least one array of photosensitive chips for acquiring image-side light field images;

[0007] A stepping control module is used to drive the sensor module to perform stepping motion along the optical axis to acquire images multiple times at different positions;

[0008] An image processing unit is used to receive multiple frames of image data and execute a light field reconstruction algorithm to generate a three-dimensional model.

[0009] Preferably, the stepping control module includes a programmable controller and a stepper motor; the programmable controller dynamically adjusts the stepping sequence according to the scene depth information, and the stepping sequence includes the number of steps, step length, and dwell time; the stepper motor is connected to the sensor module through a lead screw or linear guide to achieve precise linear movement.

[0010] Preferably, the sensor module further includes a beam splitter, which is used to split the image-side light field into multiple beams and guide them to multiple array photosensitive chips; the multiple array photosensitive chips are arranged on the beam splitting path to achieve simultaneous imaging by multiple sensors in a single operation.

[0011] Preferably, the light field reconstruction algorithm executed by the image processing unit includes calibrating and denoising multiple frames of images, calculating a disparity map using a stereo matching algorithm, converting the disparity into a depth map using triangulation principles, and generating a 3D point cloud. The stereo matching algorithm is based on block matching or feature matching, and the specific formula is as follows:

[0012] A1. Matching cost calculation: For image pairs and In pixels At this point, calculate the normalized cross-correlation (NCC) cost within the local window:

[0013]

[0014] in, Candidate values ​​for disparity. For window offset, and The mean value within the window is used as the output, which is the cost volume. , used to evaluate matching similarity;

[0015] A2. Parallax Optimization: Selecting the parallax with the lowest cost using a winner-take-all (WTA) strategy. :

[0016]

[0017] Output disparity map ;

[0018] A3. Depth Calculation: Using triangulation formulas, depth is calculated. for:

[0019]

[0020] in, Focal length The baseline distance is the sum of the sensor step distances. To account for parallax, a 3D point cloud is ultimately generated.

[0021] Preferably, it also includes an upper-level control system, which is connected to the stepping control module and the image processing unit through a communication interface. The upper-level control system is used to issue imaging commands, receive processing results, and provide a user interface for parameter setting and 3D visualization.

[0022] A method of using a three-dimensional imaging device includes the following steps:

[0023] Step S1, Equipment Calibration: The optical system and sensors are calibrated using a calibration board to determine their intrinsic and extrinsic parameters. The calibration uses the Zhang Zhengyou method to obtain the camera matrix and distortion coefficients.

[0024] Step S2, Scene Setup: Adjust the relative position of the device and the object being measured so that the object is within the measurement range. The measurement range is determined by the depth of field of the optical system. Adjust the position by previewing the image.

[0025] Step S3, Adaptive Imaging: The upper-level control system dynamically generates a stepping sequence based on scene depth estimation, controls the sensor module to step and acquire multiple frames of images. Scene depth estimation is based on the focus stack of the preview image or a TOF-assisted sensor. The stepping sequence is generated by an algorithm: depth range arrive Number of steps ,in This represents the step size corresponding to the depth resolution.

[0026] Step S4, Light Field Reconstruction: The image processing unit executes a reconstruction algorithm on multiple frames of images and outputs a three-dimensional model. Real-time performance is achieved through parallel computing.

[0027] Preferably, the adaptive imaging in step S3 includes: calculating a scene depth distribution histogram using a preview image; determining key measurement areas based on histogram peaks; optimizing the step sequence; increasing sampling points in areas with large depth variations; the preview image is acquired by a sensor at an initial position; the depth distribution histogram is estimated using either edge detection or blur analysis; and depth cues are calculated using gradient magnitude.

[0028]

[0029] Where G(x,y) represents the gradient magnitude at coordinates (x,y) in the image, reflecting the degree of change in the pixel value at that point in the horizontal and vertical directions. It is an important indicator for estimating depth cues. I represents the preview image acquired by the sensor at the initial position, which is the basic data for subsequent edge detection and blur analysis to estimate the depth distribution histogram. x is the horizontal coordinate in the image plane, and y is the vertical coordinate in the image plane, used to locate specific pixels in the image. This represents the partial derivative of the preview image I in the x-direction, describing the rate of change of the image pixel values ​​along the horizontal direction. This represents the partial derivative of the preview image I in the y-direction, describing the rate of change of the image pixel values ​​along the vertical direction. High gradient regions correspond to depth boundaries, and the step sequence sets a denser step size in these regions. The optimized step sequence is distributed in a non-uniform manner, improving data validity.

[0030] Preferably, the light field reconstruction algorithm in step S4 further includes a multi-view fusion step: extracting feature points from multiple frames of images, optimizing camera pose and 3D point coordinates through bundle adjustment to improve reconstruction accuracy. Feature point extraction uses either SIFT or ORB algorithms, and the formula for minimizing reprojection error in bundle adjustment is as follows:

[0031]

[0032] in, For the first Camera pose For three-dimensional point coordinates, For the observed image points, The projection function is optimized using the Levenberg-Marquardt algorithm to output accurate 3D point clouds.

[0033] Preferably, the above method also includes real-time feedback control, which dynamically adjusts the step sequence or camera parameters, including but not limited to exposure time and gain, based on the reconstruction results during the imaging process to optimize image quality.

[0034] Preferably, the light field reconstruction algorithm used in step S4 includes a deep learning model. This model takes multiple frames of images as input and directly outputs a depth map. During model training, synthetic light field data is used, and the loss function is the mean squared error of the depth map. The deep learning model adopts a U-Net encoder-decoder structure, with input being a tensor of stacked multiple frames of images and outputting the depth value for each pixel. During training, the loss function is:

[0035]

[0036] in, To predict depth, For true depth, The number of pixels is used to accelerate the model inference process using GPUs, enabling real-time depth estimation.

[0037] Beneficial effects

[0038] Compared with the prior art, the present invention has the following advantages:

[0039] This invention effectively improves edge accuracy by acquiring image-side light field images from different locations multiple times and utilizing a light field reconstruction algorithm, while exhibiting relatively low dependence on device components. The adaptive imaging and light field reconstruction algorithm are not significantly affected by ambient light, and the measurement range can be flexibly adjusted according to the depth of field of the optical system. By dynamically adjusting the step sequence and camera parameters through real-time feedback control, and utilizing a deep learning model for light field reconstruction, computational efficiency and matching accuracy are greatly improved, while reducing dependence on ambient lighting and texture. Attached Figure Description

[0040] Figure 1 This is a structural block diagram of the present invention;

[0041] Figure 2 This is a flowchart of the method of using the present invention;

[0042] Figure 3 This is a schematic diagram of the principle of single-sensor stepping multiple non-simultaneous imaging operations in this invention;

[0043] Figure 4 This is a schematic diagram of the principle of multi-sensor simultaneous imaging operation in this invention;

[0044] Figure 5 This is a schematic diagram illustrating the principle of multiple simultaneous imaging operations using multiple sensors in this invention.

[0045] Figure 6 This is a schematic diagram illustrating the principle of rotating reflected light field and performing multiple imaging operations in this invention. Detailed Implementation

[0046] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0047] Please see Figure 1-6 A three-dimensional imaging device, comprising:

[0048] An optical system for receiving an object-side light field and mapping it onto an image-side light field;

[0049] The sensor module includes at least one array of photosensitive chips for acquiring image-side light field images;

[0050] A stepping control module is used to drive the sensor module to perform stepping motion along the optical axis to acquire images multiple times at different positions;

[0051] An image processing unit is used to receive multiple frames of image data and execute a light field reconstruction algorithm to generate a three-dimensional model.

[0052] This invention relates to a 3D imaging device based on the principle of light field imaging. It reconstructs a 3D scene by repeatedly acquiring light field information from the image side. The device utilizes a constructed optical system and a step-able sensor module behind it. The sensor module's photosensitive chip and stepping control module are connected to a host computer. The host computer can control the stepping control module to move forward and backward, issue image acquisition commands, and acquire image data from the sensor module's photosensitive chip. During system operation, light enters the optical system from the object side and reaches the image side. The host computer controls the sensor module to step, and the photosensitive chip acquires image data with each step. After acquiring multiple frames of image data, light field data is obtained, allowing for the reconstruction of the image side's 3D model. Because the imaging system is object-image conjugate, it can also obtain external 3D information.

[0053] This invention effectively improves edge accuracy by acquiring image-side light field images from different locations multiple times and utilizing a light field reconstruction algorithm, while exhibiting relatively low dependence on device components. The adaptive imaging and light field reconstruction algorithm are not significantly affected by ambient light, and the measurement range can be flexibly adjusted according to the depth of field of the optical system. By dynamically adjusting the step sequence and camera parameters through real-time feedback control, and utilizing a deep learning model for light field reconstruction, computational efficiency and matching accuracy are greatly improved, while reducing dependence on ambient lighting and texture.

[0054] In practical applications, the 3D imaging device and its usage method of this invention can be widely used in multiple fields. In the field of autonomous driving, it can acquire high-precision, real-time 3D information about the vehicle's surrounding environment, helping the vehicle better identify obstacles, plan driving paths, and improve driving safety. In robot navigation, the device can provide robots with accurate 3D maps, enabling robots to more accurately locate and navigate in complex environments and complete various tasks. In the field of industrial inspection, it can perform rapid and accurate 3D imaging inspection of industrial products, promptly identifying surface defects and internal structural problems, improving product quality and production efficiency.

[0055] Specifically, the stepping control module includes a programmable controller (PLC) and a stepper motor. The PLC dynamically adjusts the stepping sequence based on scene depth information, and the stepping sequence includes the number of steps, step length, and dwell time. The stepper motor is connected to the sensor module via a lead screw or linear guide to achieve precise linear movement. The PLC, such as a PLC or embedded microprocessor, pre-stores multiple stepping modes or adaptively adjusts them through external input. For different scene depth information, the PLC adjusts the stepping sequence according to a preset algorithm. For example, when the scene depth changes gradually, the step length can be increased and the number of steps reduced to improve imaging speed; while when the scene depth changes drastically, the step length is reduced and the number of steps increased to ensure that enough effective image data is acquired and improve the accuracy of 3D imaging.

[0056] Specifically, the sensor module further includes a beam splitter, which divides the image-side light field into multiple beams and guides them to multiple array photosensitive chips. These multiple array photosensitive chips are arranged along the beam splitting path, enabling simultaneous imaging by multiple sensors in a single exposure. The beam splitter can employ a cubic prism or dichroic mirror to split the incident light according to wavelength or intensity. For example, a 50 / 50 beam splitter can be used to split the light into two paths, each connected to a CMOS chip. Multiple chips can be mounted on the same substrate, synchronously triggering acquisition and reducing motion artifacts. This design is suitable for high-speed dynamic scenes, acquiring multi-view images in a single exposure and improving temporal resolution.

[0057] Specifically, the light field reconstruction algorithm executed by the image processing unit includes calibrating and denoising multiple frames of images, calculating a disparity map using a stereo matching algorithm, converting the disparity into a depth map using triangulation principles, and generating a 3D point cloud. The stereo matching algorithm is based on block matching or feature matching, and the specific formula is as follows:

[0058] A1. Matching cost calculation: For image pairs and In pixels At this point, calculate the normalized cross-correlation (NCC) cost within the local window:

[0059]

[0060] in, Candidate values ​​for disparity. For window offset, and The mean value within the window is used as the output, which is the cost volume. , used to evaluate matching similarity;

[0061] A2. Parallax Optimization: Selecting the parallax with the lowest cost using a winner-take-all (WTA) strategy. :

[0062]

[0063] Output disparity map ;

[0064] A3. Depth Calculation: Using triangulation formulas, depth is calculated. for:

[0065]

[0066] in, Focal length The baseline distance is the sum of the sensor step distances. To account for parallax, a 3D point cloud is ultimately generated.

[0067] In practical applications, the accuracy and efficiency of this light field reconstruction algorithm are affected by various factors. The choice of window size is crucial for the normalized cross-correlation (NCC) cost in the matching cost calculation. Smaller windows can capture more detailed information but are more susceptible to noise interference; while larger windows are more robust to noise but may blur the detailed features of the image. Therefore, in practice, the window size needs to be selected appropriately based on the specific image characteristics and application requirements.

[0068] While the Winner-Take-All (WTA) strategy is simple and straightforward during the disparity optimization phase, it may result in some isolated erroneous disparity points in the disparity map. To further improve the quality of the disparity map, post-processing methods such as median filtering and bilateral filtering can be used to smooth the disparity map and remove these isolated erroneous points.

[0069] For depth calculation, the focal length in the triangulation formula... and baseline distance The accuracy of these parameters directly affects the precision of depth calculation. Precise calibration of both parameters is necessary before using the equipment. Additionally, due to parallax... The calculations may contain some errors, which are amplified with increasing depth. Therefore, when measuring distant objects, measures need to be taken to improve the accuracy of parallax calculations, such as increasing image resolution or using more complex stereo matching algorithms.

[0070] Furthermore, to improve the efficiency of the entire light field reconstruction process, parallel computing methods can be employed. For example, in the matching cost calculation and disparity optimization stages, the parallel computing capabilities of the graphics processing unit (GPU) can be utilized to process the calculation tasks of multiple pixels simultaneously, thereby significantly reducing the computation time.

[0071] Specifically, it also includes an upper-level control system, which is connected to the stepper control module and the image processing unit via a communication interface. This upper-level control system is used to issue imaging commands, receive processing results, and provide a user interface for parameter setting and 3D visualization.

[0072] The upper-level control system can utilize devices such as computers, tablets, or smartphones, with communication interfaces including common types like USB, Ethernet, and Wi-Fi. Users can easily set imaging parameters, such as the number of steps, step size, exposure time, and gain, through the user interface of the upper-level control system, flexibly adjusting the device's operating mode according to different application scenarios and needs. Simultaneously, users can intuitively view the 3D visualization results, observing, analyzing, and interactively operating the reconstructed 3D model.

[0073] The upper-level control system also possesses data storage and management functions, capable of storing and managing acquired image data, processed 3D models, and related imaging parameters. Users can query, retrieve, and access this data at any time, facilitating subsequent research and analysis. Furthermore, the upper-level control system can interact and share data with other external systems. For example, it can transmit 3D model data to computer-aided design (CAD) software for further design and analysis, or upload data to the cloud for storage and processing, enabling remote collaboration and data sharing.

[0074] A method of using a three-dimensional imaging device includes the following steps:

[0075] Step S1, Equipment Calibration: The optical system and sensors are calibrated using a calibration board to determine their intrinsic and extrinsic parameters. The calibration uses the Zhang Zhengyou method to obtain the camera matrix and distortion coefficients.

[0076] Step S2, Scene Setup: Adjust the relative position of the device and the object being measured so that the object is within the measurement range. The measurement range is determined by the depth of field of the optical system. Adjust the position by previewing the image.

[0077] Step S3, Adaptive Imaging: The upper-level control system dynamically generates a stepping sequence based on scene depth estimation, controls the sensor module to step and acquire multiple frames of images. Scene depth estimation is based on the focus stack of the preview image or a TOF-assisted sensor. The stepping sequence is generated by an algorithm: depth range arrive Number of steps ,in This represents the step size corresponding to the depth resolution.

[0078] Step S4, Light Field Reconstruction: The image processing unit executes a reconstruction algorithm on multiple frames of images and outputs a three-dimensional model. Real-time performance is achieved through parallel computing.

[0079] The core principle of this scheme is to reconstruct the 3D scene of the object space by acquiring image-side light field information multiple times, in a time-division or multi-path manner. Sufficient light field information is indirectly obtained through multiple samplings at the image side. Image sequences from multiple perspectives of the same object-side scene are acquired by controlling sensor position stepping, segmenting the light path, or a combination of both. The acquired multi-frame image sequence is essentially data collected by a small-scale "camera array" with known precise poses. Using stereo vision or light field reconstruction algorithms, the disparity of each pixel can be calculated, and triangulation can be used to ultimately reconstruct the depth map and 3D point cloud.

[0080] Specifically, the adaptive imaging in step S3 includes: calculating a scene depth distribution histogram using a preview image; determining key measurement areas based on histogram peaks; optimizing the step sequence; increasing sampling points in areas with large depth variations; the preview image is acquired by a sensor at an initial position; the depth distribution histogram is estimated using either edge detection or blur analysis; and depth cues are calculated using gradient magnitude.

[0081]

[0082] Where G(x,y) represents the gradient magnitude at coordinates (x,y) in the image, reflecting the degree of change in the pixel value at that point in the horizontal and vertical directions. It is an important indicator for estimating depth cues. I represents the preview image acquired by the sensor at the initial position, which is the basic data for subsequent edge detection and blur analysis to estimate the depth distribution histogram. x is the horizontal coordinate in the image plane, and y is the vertical coordinate in the image plane, used to locate specific pixels in the image. This represents the partial derivative of the preview image I in the x-direction, describing the rate of change of the image pixel values ​​along the horizontal direction. This represents the partial derivative of the preview image I in the y-direction, describing the rate of change of the image pixel values ​​along the vertical direction. High gradient regions correspond to depth boundaries, and the step sequence sets a denser step size in these regions. The optimized step sequence is distributed in a non-uniform manner, improving data validity.

[0083] Specifically, the light field reconstruction algorithm in step S4 further includes a multi-view fusion step: extracting feature points from multiple frames of images, optimizing camera pose and 3D point coordinates through bundle adjustment to improve reconstruction accuracy. Feature point extraction uses either SIFT or ORB algorithms, and the formula for minimizing reprojection error in bundle adjustment is as follows:

[0084]

[0085] in, For the first Camera pose For three-dimensional point coordinates, For the observed image points, The Levenberg-Marquardt algorithm is optimized for projection functions, outputting accurate 3D point clouds. This multi-view fusion step effectively improves reconstruction accuracy and has significant advantages in practical applications, particularly in fields with extremely high accuracy requirements.

[0086] Specifically, the above method also includes real-time feedback control, which dynamically adjusts the step sequence or camera parameters, including but not limited to exposure time and gain, based on the reconstruction results during the imaging process to optimize image quality.

[0087] For example, when the reconstruction results show that certain areas of the image are blurry or lack contrast, the real-time feedback control mechanism will automatically adjust the camera's exposure time and gain to make the image clearer and with higher contrast. If the accuracy of the reconstructed 3D model is found to be insufficient in certain parts, the system will dynamically adjust the step sequence and increase the sampling density in that area to improve reconstruction accuracy. This real-time feedback control can be flexibly adjusted according to the actual imaging situation.

[0088] Specifically, the light field reconstruction algorithm used in step S4 includes a deep learning model. This model takes multiple frames of images as input and directly outputs a depth map. During model training, synthetic light field data is used, and the loss function is the mean squared error of the depth map. The deep learning model adopts a U-Net encoder-decoder structure, with input being a tensor of stacked multiple frames of images and outputting the depth value for each pixel. During training, the loss function is:

[0089]

[0090] in, To predict depth, For true depth, The number of pixels is used to accelerate the model inference process using GPUs, enabling real-time depth estimation.

[0091] This light field reconstruction algorithm based on a deep learning model has many advantages. In practical applications, using multiple frames of images as input can fully utilize multi-angle information of the scene, resulting in an output depth map containing richer scene depth details. Synthetic light field data, as a training data source, is highly controllable, allowing for the generation of diverse training samples according to different scene requirements, thus improving the model's adaptability to various scenarios.

[0092] The mean squared error loss function accurately measures the difference between the predicted depth and the true depth, effectively guiding the model to train in a more accurate direction. The U-Net encoder-decoder structure has powerful feature extraction and recovery capabilities. The encoder can effectively extract and compress features from the stacked tensors of the input multi-frame images, converting them into abstract feature representations; the decoder can then recover the depth value of each pixel based on these feature representations, achieving good reconstruction results.

[0093] Furthermore, any content not described in detail in this specification is existing technology known to those skilled in the art.

[0094] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or three-dimensional imaging apparatus and its use that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or three-dimensional imaging apparatus and its use.

[0095] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A three-dimensional imaging device, characterized in that, include: An optical system for receiving an object-side light field and mapping it onto an image-side light field; The sensor module includes at least one array of photosensitive chips for acquiring image-side light field images; A stepping control module is used to drive the sensor module to perform stepping motion along the optical axis to acquire images multiple times at different positions; An image processing unit is used to receive multiple frames of image data and execute a light field reconstruction algorithm to generate a three-dimensional model.

2. The three-dimensional imaging device according to claim 1, characterized in that, The stepping control module includes a programmable controller and a stepper motor; the programmable controller dynamically adjusts the stepping sequence according to the scene depth information, and the stepping sequence includes the number of steps, step length and dwell time; the stepper motor is connected to the sensor module through a lead screw or linear guide to achieve precise linear movement.

3. A three-dimensional imaging device according to claim 1, characterized in that, The sensor module also includes a beam splitter, which is used to split the image-side light field into multiple beams and guide them to multiple array photosensitive chips. The multiple array photosensitive chips are arranged on the beam splitting path to achieve simultaneous imaging by multiple sensors in a single operation.

4. A three-dimensional imaging device according to claim 1, characterized in that, The light field reconstruction algorithm executed by the image processing unit includes calibrating and denoising multiple frames of images, calculating disparity maps using stereo matching algorithms, converting disparity into depth maps using triangulation principles, and generating 3D point clouds. The stereo matching algorithm is based on block matching or feature matching, and the specific formula is as follows: A1. Matching cost calculation: For image pairs and In pixels At this point, calculate the normalized cross-correlation (NCC) cost within the local window: in, Candidate values ​​for disparity. For window offset, and The mean value within the window is used as the output, which is the cost volume. , used to evaluate matching similarity; A2. Parallax Optimization: Selecting the parallax with the lowest cost using a winner-take-all (WTA) strategy. : Output disparity map ; A3. Depth Calculation: Using triangulation formulas, depth is calculated. for: in, Focal length The baseline distance is the sum of the sensor step distances. To account for parallax, a 3D point cloud is ultimately generated.

5. A three-dimensional imaging device according to claim 1, characterized in that, It also includes an upper-level control system, which is connected to the stepping control module and the image processing unit through a communication interface. The upper-level control system is used to issue imaging commands, receive processing results, and provide a user interface for parameter setting and 3D visualization.

6. A method of using a three-dimensional imaging device, characterized in that, Includes the following steps: Step S1, Equipment Calibration: The optical system and sensors are calibrated using a calibration board to determine their intrinsic and extrinsic parameters. The calibration uses the Zhang Zhengyou method to obtain the camera matrix and distortion coefficients. Step S2, Scene Setup: Adjust the relative position of the device and the object being measured so that the object is within the measurement range. The measurement range is determined by the depth of field of the optical system. Adjust the position by previewing the image. Step S3, Adaptive Imaging: The upper-level control system dynamically generates a stepping sequence based on scene depth estimation, controls the sensor module to step and acquire multiple frames of images. Scene depth estimation is based on the focus stack of the preview image or a TOF-assisted sensor. The stepping sequence is generated by an algorithm: depth range arrive Number of steps ,in This represents the step size corresponding to the depth resolution. Step S4, Light Field Reconstruction: The image processing unit executes a reconstruction algorithm on multiple frames of images and outputs a three-dimensional model. Real-time performance is achieved through parallel computing.

7. The method of using a three-dimensional imaging device according to claim 6, characterized in that, The adaptive imaging in step S3 includes: calculating a scene depth distribution histogram using a preview image; determining key measurement areas based on histogram peak values; optimizing the step sequence; increasing sampling points in areas with large depth variations; the preview image is acquired by a sensor at an initial position; the depth distribution histogram is estimated using either edge detection or blur analysis; and depth cues are calculated using gradient magnitude. Where G(x,y) represents the gradient magnitude at coordinates (x,y) in the image, reflecting the degree of change in the pixel value at that point in the horizontal and vertical directions. It is an important indicator for estimating depth cues. I represents the preview image acquired by the sensor at the initial position, which is the basic data for subsequent edge detection and blur analysis to estimate the depth distribution histogram. x is the horizontal coordinate in the image plane, and y is the vertical coordinate in the image plane, used to locate specific pixels in the image. This represents the partial derivative of the preview image I in the x-direction, describing the rate of change of the image pixel values ​​along the horizontal direction. This represents the partial derivative of the preview image I in the y-direction, describing the rate of change of the image pixel values ​​along the vertical direction. High gradient regions correspond to depth boundaries, and the step sequence sets a denser step size in these regions. The optimized step sequence is distributed in a non-uniform manner, improving data validity.

8. The method of using a three-dimensional imaging device according to claim 6, characterized in that, The light field reconstruction algorithm in step S4 also includes a multi-view fusion step: extracting feature points from multiple frames of images, optimizing camera pose and 3D point coordinates through bundle adjustment to improve reconstruction accuracy. Feature point extraction uses either SIFT or ORB algorithms. The formula for minimizing reprojection error in bundle adjustment is as follows: in, For the first Camera pose For three-dimensional point coordinates, For the observed image points, The projection function is optimized using the Levenberg-Marquardt algorithm to output accurate 3D point clouds.

9. The method of using a three-dimensional imaging device according to claim 6, characterized in that, It also includes real-time feedback control, which dynamically adjusts the step sequence or camera parameters, including but not limited to exposure time and gain, based on the reconstruction results during the imaging process to optimize image quality.

10. A method of using a three-dimensional imaging device according to claim 6, characterized in that, The light field reconstruction algorithm used in step S4 includes a deep learning model. This model takes multiple frames of images as input and directly outputs a depth map. During model training, synthetic light field data is used, and the loss function is the mean squared error of the depth map. The deep learning model adopts a U-Net encoder-decoder structure, with input being a tensor of stacked multiple frames of images and outputting the depth value for each pixel. During training, the loss function is: in, To predict depth, For true depth, The number of pixels is used to accelerate the model inference process using GPUs, enabling real-time depth estimation.

Images