A multi-spectral photometric stereo three-dimensional imaging system based on nanocrystal color conversion

Abstract

A kind of multispectral photometric stereo three-dimensional imaging system based on nanocrystal color conversion light source, including nanocrystal light source module, illumination control module, image acquisition module and three-dimensional reconstruction calculation module;Nanocrystal light source module realizes accurate illumination of multiple bands;Illumination control module realizes independent lighting of light source per channel based on shift register and single-chip microcomputer, and outputs synchronous trigger signal to camera, so that each image corresponds to a unique light source direction;Image acquisition module obtains multi-wavelength, multi-directional reflection image sequence, and uses mask to limit effective area;Three-dimensional reconstruction calculation module estimates light source direction, performs photometric stereo solving for each spectral channel, reconstructs complete three-dimensional surface topography by cross-channel normal fusion and Poisson depth recovery, and generates multispectral albedo map and pseudo-color normal map.The stability and precision of normal estimation and depth recovery under complex material surface are improved, and miniaturization, high fidelity, multi-band photometric stereo three-dimensional imaging is realized.

Description

Technical Field

[0001] This invention relates to a stereoscopic three-dimensional imaging system, specifically a multispectral photometric stereoscopic three-dimensional imaging system based on colloidal nanocrystal color conversion, belonging to the field of stereoscopic three-dimensional imaging technology. Background Technology

[0002] Photometric stereolithography is a 3D reconstruction technique that uses multiple images of a target surface acquired under different light source directions to calculate the surface normal vector and depth information by utilizing the relationship between illumination and reflection characteristics. With its advantages of being non-contact, highly accurate, and fast, this method is widely used in fields such as industrial defect detection, medical tissue imaging, cultural heritage digitization, and forensic evidence identification, becoming an important means of achieving precise surface measurement.

[0003] The core of traditional photometric stereo systems lies in illumination control and reflection modeling. Existing systems generally use fixed-band white LEDs to achieve multi-directional illumination. However, LED light sources have inherent limitations in terms of spectral range, color temperature adjustment, and illumination uniformity. These problems result in limited reflection information obtained in the reconstruction of complex surfaces (such as metals, glass, plastics, or composite materials), making it difficult to accurately model reflection behavior. Especially in the presence of specular reflection or high-reflectivity areas, specular components often overlap with diffuse reflection, interfering with albedo estimation, thus causing normal recovery errors and 3D topographic distortion. This limits the promotion and application of photometric stereo technology in demanding applications such as forensic evidence, medical imaging, and high-end industrial inspection.

[0004] Furthermore, traditional photometric stereolithography is typically based on the Lambertian reflection assumption, which assumes that the intensity of light reflected from an object's surface is proportional to the angle of incidence. However, in real-world scenarios, most surfaces are not ideal Lambertian surfaces, exhibiting complex phenomena such as specular reflection, subsurface scattering, and self-shadowing. Since the same object displays different reflective properties at different wavelengths, a single-band light source cannot provide sufficient spectral information to distinguish different reflection components, leading to incomplete information and deviations in the reconstruction results regarding texture details, geometric edges, and highlight areas.

[0005] Colloidal nanocrystals are a class of semiconductor nanomaterials capable of fluorescence color conversion. By precisely controlling their particle size and chemical composition, the emission wavelength can be continuously tuned across a broad spectral range from ultraviolet to infrared. Nanocrystalline color conversion light sources possess significant optoelectronic advantages, including narrow half-maximum width at half-maximum (HWHM) (high spectral purity), high brightness, good photostability, and high quantum efficiency. In recent years, with the development of display and lighting technologies, they have shown great potential in tunable light sources, micro-optical systems, and hyperspectral imaging.

[0006] To address the limitations of traditional light sources, such as their single and untunable spectrum, introducing nanocrystalline light sources into photometric three-dimensional systems is one approach to solving these problems. Utilizing the fluorescence color conversion of colloidal nanocrystals to construct a multispectral, narrow-band, and controllable lighting platform holds promise for overcoming the spectral aliasing and single-model limitations of traditional LED light sources.

[0007] How to adjust the emission wavelength and light source direction to accurately depict the reflection model of objects with different materials and different reflection characteristics in multiple spectral dimensions, thereby fundamentally improving the accuracy of 3D reconstruction, is a technical problem that needs to be solved in this field. Summary of the Invention

[0008] This invention addresses the problems existing in existing photometric stereo imaging technology by providing a multispectral photometric stereo imaging system based on nanocrystal color conversion that can accurately reconstruct the geometric morphology and spectral characteristics of an object's surface. This system applies multispectral photometric stereo reconstruction under nanocrystal multi-band narrowband illumination to achieve the fusion of reflection information from multiple bands of the object.

[0009] This invention relates to a multispectral photometric stereoscopic imaging system based on nanocrystalline color conversion, which employs the following technical solutions.

[0010] The system includes a nanocrystal light source module, a light control module, an image acquisition module, and a three-dimensional reconstruction calculation module. The nanocrystal light source module and the image acquisition module are both housed inside a light shield.

[0011] The nanocrystalline light source module is a multispectral nanocrystalline color conversion narrowband light source array composed of spectral channels of different wavelength groups. This array is housed within a light shield. Each nanocrystalline color conversion narrowband light source of the same wavelength constitutes a wavelength group, and all light sources within a wavelength group form a single spectral channel. The full width at half maximum (FWHM) of the emission lines of the nanocrystalline color conversion narrowband light sources in the nanocrystalline light source module is less than 25 nm. The nanocrystalline color conversion narrowband light source utilizes fluorescent semiconductor nanomaterials, including but not limited to the following categories: Group II-VI compound semiconductors (such as cadmium-based and zinc-based chalcogenides); Group III-V compound semiconductors (such as indium phosphide (InPs), indium arsenide (InAs), etc.); and perovskite quantum dots (such as halide perovskites). The multispectral nanocrystalline color conversion narrowband light source array is arranged in a ring-shaped radial pattern around the object under test. Different wavelength groups of nanocrystalline color conversion narrowband light sources are evenly distributed on the inner wall of the light shield, with azimuth angles ranging from 0° to 360°. They are layered vertically (with different elevation angle levels). Each nanocrystalline color conversion narrowband light source in the same wavelength group is evenly distributed along the same 360° azimuth angle, and the spatial number of each light source corresponds one-to-one with its wavelength number. This creates a multi-layered ring-shaped illumination structure, constructing rich illumination sampling covering multiple angles of the object's surface. This allows for the superposition of spectral sampling on top of directional sampling, achieving controllable illumination in both spatial and spectral dimensions.

[0012] The image acquisition module consists of an imaging lens and a grayscale camera (externally triggered industrial grayscale camera). The imaging lens is vertically mounted downwards at the center of the optical axis of the dome of the light shield. It is used to image the reflected light field of the object under test at different wavelengths and different incident directions onto the target surface of the grayscale camera. The grayscale camera acquires the image on the target surface (automatic acquisition controlled by a microcontroller). This module, together with the enclosed space formed by the light shield, can eliminate ambient light interference and ensure that the distribution of reflected light is determined only by the nanocrystalline light source. In actual operation, the functional implementation process of this module is as follows:

[0013] (1) Light source direction calibration: The calibration sphere reflection method is adopted. First, a high reflectivity standard sphere is placed at the center of the stage where the object to be measured is located (the center of the surface where the object to be measured is located). Then, each nanocrystalline color conversion narrowband light source in the multispectral nanocrystalline color conversion narrowband light source array is lit in sequence to obtain the bright spot image of the surface of the high reflectivity standard sphere under the illumination of each light source. According to the position of the bright spot in the image, the incident direction of the light source is inverted by the spherical reflection geometry of the high reflectivity standard sphere and the center parameter of the sphere (since the geometry and reflection characteristics of the surface of the high reflectivity standard sphere are known, the position of the bright spot and the incident direction have a fixed geometric relationship). The set of light source direction vectors is calculated, and all light source direction vectors are normalized according to the numbering order of the nanocrystalline color conversion narrowband light sources and output to the light source direction matrix file.

[0014] (2) Image sequence acquisition: Multiple (e.g., 5) illumination images from different directions are acquired in each spectral channel using a grayscale camera to obtain a sequence of reflection images of the target surface.

[0015] (3) Data preprocessing: Read the light source direction matrix file and robustly sort and map the acquired image files according to the numerical index to ensure that the image data corresponds one-to-one with the calibrated light source direction sequence; at the same time, use the mask file to limit the effective reconstruction area to eliminate background interference and improve the effective signal-to-noise ratio, providing standardized input data for subsequent albedo and normal solving;

[0016] The illumination control module is used for channel-level switching and timing management of the multispectral channel nanocrystalline color conversion narrowband light source array, and for hardware-level synchronous triggering with the grayscale camera in the image acquisition module. Acquisition is performed sequentially according to the following order: lighting one light source in a single spectral channel, waiting for stabilization, triggering the grayscale camera, turning off the light source, and switching to the next light source. After completion within the same spectral channel, the module switches to the next spectral channel. The preferred timing parameters are: stabilization time of the nanocrystalline color conversion narrowband light source 50ms, camera trigger pulse width 50ms, and adjacent light source switching interval 50ms. This module supports strategies such as multi-channel expansion, group lighting, and constant-on indication to adapt to different exposure and target material conditions, and achieves highly reliable channel switching by sending bitmap control via a bus.

[0017] The 3D reconstruction calculation module estimates the light source direction based on the calibrated spherical reflectance method and performs photometric stereo solutions for each spectral channel. It then reconstructs the complete 3D surface topography through cross-channel normal fusion and Poisson depth, generating a multispectral albedo map and a pseudo-color normal map. The specific implementation process is as follows:

[0018] The algorithm reads all spectral channel image sequences and their corresponding light source directions, and normalizes the direction vectors. Before photometric stereo reconstruction, a mask is constructed to limit the effective calculation area to exclude background, shadows, or non-target regions. Median normalization is performed on each frame within the mask to reduce inter-frame brightness drift. Then, a multispectral photometric stereo algorithm is used to solve for each spectral channel. Initial values ​​are obtained using the least squares method, and then the brightest frames are discarded at the pixel level to suppress specular reflection interference before a weighted solution is performed to obtain the surface albedo and normal vector of that spectral channel. Based on this, the normal vectors of each spectral channel are pixel-by-pixel superimposed and normalized to form a fused image. A valid vector field is used to improve the stability and consistency of normal estimation. In the depth recovery stage, a sparse linear equation system with consistent surface gradient is constructed based on the fused normal vector field, and a pre-conditioned conjugate gradient solver is used for regularization to obtain the depth distribution of the target surface and display it in a normalized manner according to the mask range. At the same time, the module selects spectral channel data of three bands (640nm, 540nm and 470nm, corresponding to R, G and B) and normalizes them according to their respective maximum values ​​to combine them into a synthetic albedo map. Finally, the output includes the albedo and normal map of each spectral channel, the fused pseudo-color display normal, the synthetic albedo map and the depth map, and the multispectral photometric stereo 3D reconstruction result.

[0019] The mask generation adopts an automatic segmentation method based on brightness distribution and morphological screening. First, a single frame image of the scene is read to obtain single-channel brightness information. Then, the grayscale image is binarized and segmented using an adaptive thresholding method to automatically distinguish the target area from the background area. The connected region with the largest number of pixels is automatically selected as the main target area to ensure that the mask covers the main object being measured. Finally, the mask is displayed and saved in the form of a black and white binary image.

[0020] The nanocrystalline light source module, illumination control module, and image acquisition module are all uniformly controlled by a driver (such as an Arduino driver) and control circuit. The driver and control circuit is used to precisely and independently illuminate each spectral channel of the nanocrystalline light source array sequentially according to its number. After illumination and stabilization, a rising edge pulse is sent to the grayscale camera to trigger exposure, achieving strict synchronization between illumination and acquisition. Taking the Arduino driver and control circuit as an example, it includes an Arduino microcontroller and two cascaded shift registers (74HC595). The microcontroller is connected to the cascaded shift registers via a serial control pin. The parallel outputs of the shift registers are connected to the driver terminals of each spectral channel of the nanocrystalline light source array to expand the number of control channels. The microcontroller's synchronization signal output pin is connected to the external trigger interface on the grayscale camera. Under the unified control of the microcontroller, after the nanocrystalline light source is illuminated and stabilized sequentially according to its number, a rising edge pulse is sent to the camera to trigger exposure, achieving strict synchronization between illumination and acquisition.

[0021] This invention centers on the fusion of active illumination from a nanocrystalline color conversion light source and a photometric stereo algorithm. Based on multi-band illumination information, it utilizes the photometric stereo algorithm to achieve spatial geometry reconstruction, constructing a multispectral 3D imaging system with spectral selectivity and spatial topography reconstruction capabilities. The system solution encompasses multi-band controllable illumination, light source orientation calibration, reflection characteristic modeling, and multi-channel fusion reconstruction. By flexibly adjusting the emission wavelength and light source direction, this technology can precisely characterize the reflection model of objects with different materials and reflection characteristics across multiple spectral dimensions. Through high-precision albedo estimation and normal recovery, it fundamentally improves the accuracy of 3D reconstruction.

[0022] This invention combines multi-band active illumination with a robust photometric stereo algorithm to acquire multispectral reflectance information while maintaining high spatial resolution, thereby accurately reconstructing the geometric shape and spectral characteristics of an object's surface. It has the following features:

[0023] 1. Multi-band active photometric stereo reconstruction was achieved;

[0024] 2. It enriches the information on the surface albedo of the measured object;

[0025] 3. By utilizing the narrow bandgap and spectral tunability of nanocrystalline light sources, multispectral reflection constraints are introduced, which improves the stability and accuracy of normal estimation and depth recovery on complex material surfaces.

[0026] 4. The size of the spectral imaging system has been reduced, enabling miniaturized, high-fidelity, multi-band photometric stereoscopic imaging. Attached Figure Description

[0027] Figure 1 This is a schematic diagram of the principle of the multispectral photometric three-dimensional imaging system based on nanocrystalline color conversion of the present invention.

[0028] Figure 2 This is a schematic diagram of the structural composition of the photometric stereoscopic imaging system based on nanocrystalline color conversion according to the present invention.

[0029] Figure 3 This is an example of the wavelength and direction distribution diagram of the light source array in this invention.

[0030] Figure 4 This is a graphical representation of the reconstruction algorithm in this invention.

[0031] In the diagram: 1. The object under test, 2. A light shield, 3. An externally triggered industrial grayscale camera, 4. A nanocrystal light source array, 5. An Arduino driver and control circuit, and 6. A computer. Detailed Implementation

[0032] This invention proposes a multispectral photometric stereo imaging system based on nanocrystalline color conversion. The core design of this system lies in combining multi-band active illumination with a robust photometric stereo algorithm to achieve the acquisition of multispectral reflectance information while ensuring high spatial resolution, thereby accurately reconstructing the geometric morphology and spectral characteristics of an object's surface.

[0033] Figure 1 This paper presents a general research framework for a multispectral photometric stereo imaging system based on nanocrystalline color conversion. To achieve hyperspectral 3D reconstruction without relying on complex multi-camera arrays or adjustable filtering components, this invention uses the fusion of nanocrystalline color conversion active illumination and photometric stereo algorithms as its core concept. Based on multi-band illumination information, the photometric stereo algorithm achieves spatial geometry restoration, constructing a multispectral 3D imaging system with spectral selectivity and spatial topography reconstruction capabilities. The system solution is built from aspects such as multi-band controllable illumination, light source orientation calibration, reflection characteristic modeling, and multi-channel fusion reconstruction.

[0034] The implementation process of this invention can be briefly summarized as follows:

[0035] 1. Establish a multispectral nanocrystalline color conversion narrowband light source array to achieve multi-band controllable illumination;

[0036] 2. The direction of the light source is determined by the standard spherical reflection method, and an accurate geometric model is established;

[0037] 3. Acquire multi-angle, multi-spectral reflectance images and generate masked regions;

[0038] 4. Calculate the normal vector, albedo, and residual map based on the photometric stereo algorithm;

[0039] 5. Perform multi-channel normal fusion and depth restoration to output three-dimensional topography results.

[0040] Starting from photometric stereo imaging technology, this invention constructs a multispectral photometric stereo 3D imaging system based on a nanocrystalline color conversion light source, comprising a nanocrystalline light source module, an illumination control module, an image acquisition module, and a 3D reconstruction calculation module. The nanocrystalline light source module employs multi-channel narrowband nanocrystalline light-emitting units to achieve precise multi-band illumination. The illumination control module, based on a shift register and a microcontroller, enables independent channel-by-channel illumination of the light source and outputs a synchronous trigger signal to the camera, ensuring that each image corresponds to a unique light source direction. The image acquisition module acquires multi-wavelength, multi-directional reflection image sequences under shaded conditions and uses a mask to define the effective area. The 3D reconstruction calculation module estimates the light source direction based on the calibrated sphere reflection method, performs photometric stereo solutions for each spectral channel, reconstructs the complete 3D surface topography through cross-channel normal fusion and Poisson depth recovery, and generates a multispectral albedo map and a pseudo-color normal map. First, all channel image sequences and corresponding orientation matrices are read. Masking and brightness normalization are performed on the images. Then, a multispectral photometric stereo algorithm is used to obtain the surface albedo and normal vector by channel. Next, the normal vectors of each channel are superimposed pixel by pixel and normalized to form a fused normal vector field. Finally, a set of surface gradient consistency equations is constructed based on the fused normal vector field and solved by the preconditional conjugate gradient method to obtain the depth distribution of the target surface, thus forming a multispectral photometric stereo 3D reconstruction result containing albedo, normal, and depth information.

[0041] See Figure 2 The system comprises an externally triggered industrial grayscale camera (3), a nanocrystalline light source array (4), an Arduino driver and control circuit (5), and a computer (6). The computer (6) handles data processing, and the components work together to perform active imaging and 3D reconstruction under multi-band illumination. In terms of hardware connectivity, the nanocrystalline light source array (4) is connected to the Arduino driver and control circuit (5), which precisely and independently controls the illumination of each spectral channel in the array. Both the grayscale camera (3) and the Arduino driver and control circuit (5) are connected to the computer (6) via data interfaces, enabling data transmission and command issuance.

[0042] The nanocrystalline light source module 4 is a multispectral nanocrystalline color conversion narrowband light source array composed of spectral channels of different wavelengths. The spectral channels can be 9, 10, or more, including narrowband channels with center wavelengths of 640nm, 620nm, 590nm, 570nm, 540nm, 520nm, 470nm, and 410nm. It should be noted that this invention is not limited to the specific wavelength values ​​or number of channels mentioned above. Depending on the material properties of the object being measured and the application requirements, different numbers (e.g., increasing to 10 channels) and different peak wavelengths of nanocrystalline light-emitting units can be flexibly configured to obtain richer spectral reflectance information. The nanocrystalline color conversion narrowband light source uses fluorescent semiconductor nanomaterials, including but not limited to the following categories: Group II-VI compound semiconductors (such as cadmium-based zinc-based chalcogenides); Group III-V compound semiconductors (such as indium phosphide (InPs), indium arsenide (InAs), etc.); perovskite quantum dots (such as halide perovskites). In terms of microstructure, the nanocrystalline materials encompass various forms such as quantum dots, one-dimensional nanorods, and two-dimensional nanosheets (or quantum wells). By utilizing the quantum confinement effect of these materials, and by controlling their size, composition, and morphology, continuous and precise control of the emission wavelength can be achieved.

[0043] In the nanocrystalline light source array 4, narrowband nanocrystalline color conversion light sources of different wavelengths are uniformly distributed on the inner wall of the light shield 2. The light shield 2 is used to create a closed optical imaging environment, isolating natural light, ambient scattered light, and external reflection interference, ensuring that the nanocrystalline light source is the only effective illumination source. Its inner wall is treated with an anti-reflection coating to reduce internal reflection and avoid interference in the image.

[0044] Multiple light sources with different emission wavelengths are set up based on the nanocrystalline light source array 4. The spatial position and illumination direction of each light source are determined according to the hemispherical coordinate system and corrected in the subsequent calibration process. During operation, each light source in the nanocrystalline light source array 4 is lit sequentially to provide narrowband monochromatic light illumination from different incident directions to the object under test 1. The high spectral purity and low divergence angle of the narrowband nanocrystalline color conversion enable it to produce reflection features that are easier to separate and model on complex material surfaces.

[0045] Figure 3The light source direction distribution of the nanocrystalline light source array 4 is presented. An example is given using eight channels with center wavelengths of 640nm, 620nm, 590nm, 570nm, 540nm, 520nm, 470nm, and 410nm. In the specific implementation, the light source direction calibration is achieved using the calibration sphere reflection method. A highly reflective standard sphere is placed in the experimental scene, and each nanocrystalline light source channel in the array is illuminated sequentially to obtain a bright spot image on the surface of the sphere under illumination by each light source. Computer 6 automatically reads the sequentially captured images, detects and extracts the maximum brightness point and its neighborhood centroid in the bright spot area, thereby determining the highlight position coordinates of the light source on the calibration sphere. Combining the coordinates of the center of the calibration sphere and its radius parameters, the direction of the spherical normal vector corresponding to this point is calculated. Based on the relationship between the spherical normal and the imaging optical axis, the reflection geometry determines the incident light direction, that is, the mapping from the position of the bright spot to the light source direction is achieved through the specular reflection characteristics of the spherical surface. After calibration, all light source direction vectors are normalized to ensure that the length of each direction quantity is consistent. The final output light source direction matrix is ​​saved as a light source direction sequence text file in order of light source number. This file contains the three-dimensional direction components of each light source in the imaging coordinate system. The entire nanocrystalline light source array 4 is arranged in a ring-radial pattern centered on the object under test 1. Light sources of the same wavelength are grouped and uniformly distributed along a 360° azimuth angle. Figure 3 As shown in the upper left section, each individual nanocrystalline light-emitting unit corresponds to a unique emission direction, with its direction vector pointing towards the center of the hemispherical light shield, forming a complete horizontal ring illumination sampling, which allows for sufficient azimuth change information to be obtained during photometric stereo reconstruction. Figure 3 The lower left section further illustrates the pitch angle distribution of each light source. Different colored arrows represent different pitch angle levels, and the light sources form a multi-layered distribution in the vertical direction, improving the stability during 3D reconstruction. Figure 3 The table on the right shows the correspondence between the direction of each light source and its emission wavelength. The light sources are numbered sequentially according to their direction and correspond one-to-one with the wavelength segments. This invention further superimposes the spectral distribution on the basis of the spatial directional distribution, so that each direction contains both geometric illumination information and spectral information of different wavelengths, providing accurate input conditions for subsequent multi-channel photometric stereo normal vector estimation and material albedo analysis.

[0046] Both the illumination control module and the image acquisition module are implemented through the Arduino driver and control circuit 5 for unified control. The nanocrystalline light source module 4 is set inside the light shield 2. After illuminating the object under test 1 with nanocrystalline light sources of different wavelengths in a light-shielding environment, the grayscale camera 3 collects the reflected light from the object under test 1 to obtain the final image sequence for subsequent calculations.

[0047] The object under test 1 is placed at the center of a hemispherical light shield 2, and its surface produces an angle-dependent distribution of reflected light under illumination from nanocrystalline light sources in different directions. A grayscale camera 3 is mounted vertically downwards at the center of the optical axis at the top of the light shield 2 to receive the reflected light field from the object under test 1 and optically image it onto the target surface of the grayscale camera 3. The focal length and mounting position of the lens are pre-adjusted to ensure that the entire effective illumination area is within the clear imaging range, while suppressing distortion and ensuring that the pixel gradient information on which subsequent photometric stereo calculations depend is accurate and effective.

[0048] The Arduino driver and control circuit 5 is used to precisely and independently control the illumination of each spectral channel of the nanocrystalline light source array 4. The Arduino driver and control circuit 5 includes an Arduino microcontroller and two cascaded shift registers, wherein the shift registers are 74HC595 chips. The serial data output, clock output, and latch control terminals of the microcontroller are respectively connected to the serial data input, shift clock, and latch terminals of the first 74HC595; the serial data output of the first 74HC595 is connected to the serial data input of the second 74HC595, thus forming a cascaded structure to expand the number of parallel control output channels. The parallel output terminals of the two 74HC595 chips are respectively connected to the drive input terminals of the corresponding spectral channel light-emitting units in the nanocrystalline light source array 4. Each parallel output terminal corresponds to one spectral channel light-emitting unit, and the individual, independent illumination and deactivation control of each spectral channel light-emitting unit is achieved through programming. The microcontroller sends a bitmap sequence to the shift register via data, clock, and latch terminals. Each transmission corresponds to the conduction state of a specific channel in the nanocrystalline light source. Upon receiving the bit sequence from the microcontroller, the shift register latches the high and low voltage states of the corresponding positions and outputs them to the nanocrystalline light source driver, enabling the on / off control of any one or more light sources. This bitmap-based shift register control method allows multiple nanocrystalline light sources to be lit independently and sequentially according to their channel numbers. Simultaneously, the microcontroller generates a camera trigger signal that strictly corresponds to the light source illumination sequence. Specifically, after a light source is turned on and a set brightness stabilization delay has elapsed, the microcontroller outputs a fixed-width rising edge pulse at the grayscale camera's external trigger terminal, causing the grayscale camera 3 to complete an exposure under that light source illumination. Upon receiving the trigger signal, the grayscale camera 3 immediately records the reflected image under the current lighting conditions, ensuring that each captured image uniquely corresponds to a single light source direction. Through the above structure, the Arduino driver and control circuit 5 realizes the automated execution of light source conduction, stable timing control, camera synchronous triggering, and multi-channel sequential scanning, providing image data with stable timing and clear light source correspondence for photometric stereo reconstruction.

[0049] Computer 6 performs core functions such as light source orientation calibration, mask generation, photometric stereo solution, and depth recovery. Computer 6 reads the image sequence of all spectral channels transmitted by grayscale camera 3, generates a light source orientation matrix using the calibration sphere reflection method, and defines the effective calculation area based on the mask. Subsequently, the computer executes a multispectral photometric stereo algorithm in each channel to calculate the corresponding albedo distribution and normal vector field, and performs normal fusion and Poisson integral on the channel results to reconstruct the target object's two-dimensional albedo map, pseudo-color normal map, and depth information.

[0050] The computer implements the 3D reconstruction calculation module, reading all spectral channel image sequences and corresponding light source directions, and normalizing the direction vectors. Before photometric stereo reconstruction, a mask is constructed to limit the effective calculation area to exclude background, shadows, or non-target areas. Median normalization is performed on each frame within the mask to reduce inter-frame brightness drift. Subsequently, a multispectral photometric stereo algorithm is used to solve for each spectral channel. Initial values ​​are obtained using the least squares method, and then the brightest frames are discarded at the pixel level to suppress specular reflection interference before a weighted solution is performed to obtain the surface albedo and normal vector of that spectral channel. Based on this, the normal vectors of each spectral channel are pixel-by-pixel superimposed and single-dimensionally reconstructed. The system is processed to form a fused normal vector field, which improves the stability and consistency of the normal estimation. In the depth recovery stage, a sparse linear equation system with consistent surface gradient is constructed based on the fused normal vector field, and a pre-conditioned conjugate gradient solver is used for regularization to obtain the depth distribution of the target surface and display it in a normalized manner according to the mask range. At the same time, the module selects spectral channel data of three bands (640nm, 540nm and 470nm, corresponding to R, G and B) and normalizes them according to their respective maximum values ​​to combine them into a synthetic albedo map. Finally, the output includes the albedo and normal map of each spectral channel, the fused pseudo-color display normal, the synthetic albedo map and the depth map, and the multispectral photometric stereo 3D reconstruction result.

[0051] Figure 4This is a graphical representation of the 3D reconstruction algorithm used in this invention. In the calculation, the spatial resolution of the target object is assumed to be n×n. During reconstruction, the surface height can be represented as a vector of length n². The entire system has c spectral channels. Each spectral channel acquires m reflection images from different incident directions, thus forming cm brightness observation frames under all band and direction combinations. For the j-th spectral channel, its m brightness observations can be represented as Ij∈R^(m×n²), where each column corresponds to the brightness value of a pixel under different light source directions. The light source direction matrix corresponding to this channel is assumed to be Lj∈R^(m×3), and each row of Lj has been normalized to a unit vector during the calibration stage. Based on the Lambertian reflection model, there is a linear relationship between brightness and surface normal, which can be expressed as: Ij=Lj·Gj, where Gj∈R^(3×n²) is the normal coefficient matrix of this channel before normalization. To suppress overly bright observations caused by specular reflection, this invention removes the brightest frame from the brightness vector of each pixel before solving Gj, and constructs a corresponding weight matrix Wj to filter brightness observations, thereby forming a constraint subset that better conforms to the Lambert model.

[0052] Subsequently, by solving for the weighted least squares of Ij and Lj, the albedo αj and normal vector Nj of channel j can be obtained. Multispectral fusion of the albedo maps of each spectral channel yields a similar result. Figure 4 (b) RGB combined albedo map. The fusion process normalizes the reflection intensity of different bands according to wavelength and maps it to the RGB space, highlighting material differences in the color dimension. By superimposing and normalizing the normal vectors of all channels at each pixel position, the fused normal vector field can be obtained. Figure 4 (c) The pseudo-color normal map maps the three components of the normal to the three RGB channels respectively, making the changes in the direction of the spatial gradient intuitive.

[0053] After obtaining the fused normal vector field, this invention constructs a depth constraint equation to solve for the morphology of the target surface. The normal vector provides partial derivative information of the local surface, i.e.: ; By establishing a consistency constraint between the height difference of adjacent pixels and the aforementioned gradient across the entire mask region, a sparse linear equation system can be formed. Its overall structure can be abstractly represented as: Mz = b, where M is a sparse coefficient matrix reflecting the gradient constraint relationship between pixels; z is the unknown surface height vector after flattening by rows and columns; and b is the gradient observation derived from the fused normal vector. Since the matrix M typically has hundreds of thousands of dimensions and strong sparsity, this invention employs the preconditioned conjugate gradient method to solve the above normal equations. The preconditioned conjugate gradient method introduces a preconditioner during the iteration process, allowing the solution to be performed in a transformation space with significantly improved condition numbers, thereby reducing the number of iterations and improving convergence efficiency. For matrix M... TGiven the structural characteristics of M, this invention employs incomplete Cholesky decomposition as the preconditioning method. Under this transformation, the conjugate gradient search direction can more closely approximate the optimal descent direction, resulting in higher error decay efficiency in each iteration. Compared to the conjugate gradient method without a preconditioner, the incomplete Cholesky preconditioner used in this invention can significantly reduce the number of iterations at typical depth reconstruction scales, achieving stable acceleration.

[0054] After the solution is completed, the depth vector z is rearranged into an n×n spatial height distribution, and the relative height map of the target object is obtained by translation and normalization. Figure 4 The "Depth Map and 3D Shape Rendering" at the bottom shows the complete 3D structure obtained from the photometric stereo reconstruction, and its concave and convex details and surface orientation are basically consistent with the actual object.

Claims

1. A multispectral photometric stereoscopic imaging system based on nanocrystalline color conversion, characterized in that, It includes a nanocrystal light source module, a light control module, an image acquisition module, and a 3D reconstruction calculation module. The nanocrystal light source module and the image acquisition module are both housed inside a light shield. The nanocrystalline light source module is a multispectral nanocrystalline color conversion narrowband light source array composed of spectral channels of different wavelength groups. This array is set inside a light shield. Each nanocrystalline color conversion narrowband light source of the same wavelength constitutes a wavelength group, and all the light sources in a wavelength group constitute a spectral channel. Each independent nanocrystalline color conversion narrowband light source corresponds to a unique emission direction, and its direction vector points to the center of the light shield. When working, each light source is lit up sequentially to provide narrowband monochromatic light illumination from different incident directions to the object under test. The image acquisition module consists of an imaging lens and a grayscale camera. The imaging lens is vertically mounted downwards at the center of the optical axis of the dome of the light shield. It is used to estimate the incident direction of the light source based on the calibration sphere reflection method, and to image the reflected light field of the object under different wavelengths and different incident directions of the light source onto the target surface of the grayscale camera. The grayscale camera acquires the image on the target surface. The illumination control module is used to perform channel-level switching and timing management of the multispectral channel nanocrystalline color conversion narrowband light source array, and to perform hardware-level synchronous triggering with the grayscale camera in the image acquisition module. The acquisition is performed sequentially according to the order of lighting up one light source in a single spectral channel, waiting for stabilization, triggering the grayscale camera, turning off the light source, and switching to the next light source. After the acquisition is completed in the same spectral channel, the next spectral channel is switched. The three-dimensional reconstruction calculation module performs photometric stereo solution for each spectral channel; it performs pixel-by-pixel superposition and normalization of the normal vectors of each spectral channel to form a fused normal vector field; in the depth recovery stage, it constructs a sparse linear equation system with consistent surface gradient based on the fused normal vector field, and uses a pre-conditioned conjugate gradient solver for regularization to obtain the depth distribution of the target surface, thereby reconstructing the three-dimensional surface morphology and generating a multispectral albedo map and a pseudo-color normal map.

2. The multispectral photometric three-dimensional imaging system based on nanocrystalline color conversion according to claim 1, characterized in that, The full width at half maximum (FWHM) of the emission spectral lines of the nanocrystalline color conversion narrowband light source in the nanocrystalline light source module is less than 25 nm.

3. The multispectral photometric stereoscopic imaging system based on nanocrystalline color conversion according to claim 1, characterized in that, The nanocrystalline color conversion narrowband light source uses fluorescent semiconductor nanomaterials, including but not limited to the following categories: group II-VI compound semiconductors, group III-V compound semiconductors, and perovskite quantum dots.

4. The multispectral photometric three-dimensional imaging system based on nanocrystalline color conversion according to claim 1, characterized in that, The multispectral nanocrystalline color conversion narrowband light source array is arranged in a ring-radial pattern with the object under test as the center. The nanocrystalline color conversion narrowband light sources of different wavelength groups are evenly distributed on the inner wall of the light shield, and are evenly distributed in the ring azimuth angle from 0° to 360°. They are arranged in layers in the vertical direction. Each nanocrystalline color conversion narrowband light source of the same wavelength group is evenly distributed in the same layer in the 360° azimuth angle direction. The spatial number of each nanocrystalline color conversion narrowband light source corresponds one-to-one with the wavelength number.

5. The multispectral photometric three-dimensional imaging system based on nanocrystalline color conversion according to claim 1, characterized in that, The function implementation process of the image acquisition module is as follows: (1) Calibration of incident direction of light source: The calibration sphere reflection method is adopted. First, a high reflectivity standard sphere is placed at the center of the stage where the object is located. Then, each nanocrystalline color conversion narrowband light source in the multispectral nanocrystalline color conversion narrowband light source array is lit in sequence to obtain the bright spot image on the surface of the high reflectivity standard sphere under the illumination of each light source. According to the position of the bright spot in the image, the incident direction of the light source is inverted by the spherical reflection geometry of the high reflectivity standard sphere and the center parameter of the sphere. The set of light source direction vectors is calculated and all light source direction vectors are normalized according to the numbering order of the nanocrystalline color conversion narrowband light source and output to the light source direction matrix file. (2) Image sequence acquisition: Illumination images from different directions are acquired in each spectral channel using a grayscale camera to obtain a sequence of reflection images of the target surface; (3) Data preprocessing: Read the light source direction matrix file and robustly sort and map the acquired image files according to the numerical index to ensure that the image data corresponds one-to-one with the calibrated light source direction sequence; at the same time, use a mask to limit the effective reconstruction area to eliminate background interference and improve the effective signal-to-noise ratio, providing standardized input data for subsequent albedo and normal calculation.

6. The multispectral photometric stereoscopic imaging system based on nanocrystalline color conversion according to claim 1, characterized in that, The timing parameters in the illumination control module are as follows: stabilization time of the nanocrystalline color conversion narrowband light source is 50ms, camera trigger pulse width is 50ms, and the switching interval between adjacent light sources is 50ms.

7. The multispectral photometric stereoscopic imaging system based on nanocrystalline color conversion according to claim 5, characterized in that, The mask is generated using an automatic segmentation method based on brightness distribution and morphological screening. First, a single-frame image of the acquisition scene is read to obtain single-channel brightness information. Subsequently, the grayscale image is binarized using an adaptive thresholding method to automatically distinguish the target area from the background area. The connected region with the largest number of pixels is automatically selected as the main target area, thereby ensuring that the mask covers the main object being measured. The final mask is displayed and saved in the form of a black and white binary image.

8. The multispectral photometric stereoscopic imaging system based on nanocrystalline color conversion according to claim 1, characterized in that, The specific implementation process of the three-dimensional reconstruction calculation module is as follows: read all spectral channel image sequences and corresponding light source directions and perform normalization processing on the direction vectors; before photometric stereo reconstruction, construct a mask to limit the effective calculation area to exclude background, shadow or non-target areas; and perform median normalization on each frame image within the mask to reduce inter-frame brightness drift. Subsequently, a multispectral photometric stereo algorithm is used to solve for each spectral channel. First, the initial value is obtained using the least squares method. Then, the brightest frames are discarded at the pixel level to suppress specular reflection interference, and the solution is weighted again to obtain the surface albedo and normal vector of the spectral channel. On this basis, the normal vectors of each spectral channel are superimposed and normalized pixel by pixel to form a fused normal vector field, which is used to improve the stability and consistency of the normal estimation. In the depth recovery stage, a sparse linear equation system with consistent surface gradient is constructed based on the fused normal vector field, and a pre-conditioned conjugate gradient solver is used for regularization to obtain the depth distribution of the target surface, which is then normalized and displayed according to the mask range. At the same time, spectral channel data from three bands, namely 640nm, 540nm and 470nm, are selected, normalized according to their respective maximum values, and then combined into a synthetic albedo map. The final output includes the albedo and normal map of each spectral channel, the fused pseudo-color display normal, the synthetic albedo map and the depth map, and the multispectral photometric stereoscopic 3D reconstruction result.

9. The multispectral photometric stereoscopic imaging system based on nanocrystalline color conversion according to claim 1, characterized in that, The nanocrystal light source module, illumination control module, and image acquisition module are all controlled by a unified drive and control circuit. The drive and control circuit is used to precisely and independently illuminate each spectral channel of the nanocrystal light source array according to its number. After the light source is illuminated and stabilized, a rising edge pulse is sent to the grayscale camera to trigger exposure, thus achieving strict synchronization between illumination and acquisition.

10. The multispectral photometric stereoscopic imaging system based on nanocrystalline color conversion according to claim 9, characterized in that, The driving and control circuit includes a microcontroller and two cascaded shift registers. The microcontroller is connected to the cascaded shift registers via a serial control pin. The parallel output terminals of the shift registers are respectively connected to the driving terminals of each spectral channel of the nanocrystal light source array to expand the number of control channels. The synchronization signal output pin of the microcontroller is connected to the external trigger interface on the grayscale camera.

Images