An underwater co-aperture three-dimensional fluorescence-reflectance imaging system and method

Abstract

The application discloses an underwater common-aperture three-dimensional fluorescence-reflectivity imaging system and method, and relates to the technical field of underwater optical imaging and detection. The system comprises: a detection camera which is a waterproof pressure cabin body, and is internally provided with a three-dimensional imaging camera and a fluorescence imaging camera, shares the same glass window, and splits the reflected light and fluorescence from a measured object through a dichroic mirror to divide the light path into a three-dimensional imaging camera light path and a fluorescence imaging camera light path, and synchronously collects three-dimensional morphology and fluorescence information under the same optical aperture; and a projection system which is used for projecting a structured light fringe to the measured object. The application adopts a common-aperture split imaging design, splits the reflected light and fluorescence from the same optical aperture into two paths through the dichroic mirror, and synchronously receives the two paths by the three-dimensional imaging camera and the fluorescence imaging camera respectively, so that the consistency in space and the synchronism in time of three-dimensional point clouds and fluorescence images are realized, and the overall precision and usability of imaging data are significantly improved.

Description

Technical Field

[0001] This invention relates to the field of underwater optical imaging and detection technology, specifically to an underwater co-aperture three-dimensional fluorescence-reflectivity imaging system and method. Background Technology

[0002] With the rapid development of underwater exploration, ecological environment monitoring and resource surveys, the demand for high-precision underwater visualization technology is increasing. Underwater imaging, as an important means of acquiring underwater information, plays an increasingly important role. Among them, active optical underwater imaging systems actively transmit optical modulation signals to the target being measured. Compared with acoustic imaging, it can provide higher resolution three-dimensional information in the underwater environment, thereby helping relevant technicians to conduct underwater investigations and surveys.

[0003] In existing technologies, in complex aquatic environments, traditional imaging methods cannot simultaneously achieve three-dimensional topography reconstruction, reflectivity imaging, and fluorescence imaging under a single optical path, leading to difficulties in data registration and low imaging efficiency. At the same time, the absorption and scattering of light by water can seriously affect the modulation of projected fringes, the uniformity of fluorescence excitation, and the accuracy of reflectivity and fluorescence signals, thereby affecting the accuracy of three-dimensional reconstruction and spectral information restoration. Therefore, how to achieve efficient synchronous acquisition of three-dimensional point clouds, reflectivity, and fluorescence images, and effectively improve the realism and reliability of imaging through signal compensation based on a water attenuation model, is the problem that this invention aims to solve. To this end, an underwater co-aperture three-dimensional fluorescence-reflectivity imaging system and method are proposed. Summary of the Invention

[0004] To solve the above-mentioned technical problems, the present invention provides the following technical solution: Firstly, an underwater co-aperture three-dimensional fluorescence-reflectivity imaging system, comprising: The detection camera has a common aperture beam-splitting imaging component installed in its camera compartment. It is used to receive reflected light and fluorescence from the object being measured. It also uses a dichroic mirror to split the light path from the same optical aperture into a three-dimensional imaging camera light path and a fluorescence imaging camera light path. It can simultaneously acquire three-dimensional morphology and fluorescence information under the same optical aperture, avoid multi-path registration problems, and improve imaging efficiency and data consistency. The camera calibration module is used to calibrate the intrinsic parameters of the 3D imaging camera and the fluorescence imaging camera in the detection camera using the Zhang Zhengyou calibration method, to calibrate the intrinsic parameters of the projection system using the inverse camera model of the projector, and to obtain the extrinsic parameters between the 3D imaging camera, the fluorescence imaging camera and the projection system through multi-view joint calibration. The projection end flat field calibration module is used to ensure the purity of the excitation spectrum by using a single-wavelength LED light source and a narrow-band filter in the projection system, and to calibrate the uniformity of the projection grayscale in the air through flat field calibration technology to ensure the uniformity of fluorescence excitation of underwater targets. At the same time, the calibration parameters are used to calculate the fluorescence intensity and target reflectivity. The synchronous acquisition module is used to project multi-frequency sinusoidal fringes underwater by a projection system, with three high-frequency and three low-frequency fringes. The three-dimensional imaging camera simultaneously acquires six fringe deformation images, uses the multi-frequency heterodyne method to solve the phase to obtain the absolute phase, and calculates the three-dimensional point cloud of the object under test by combining the calibration parameters. At the same time, the fluorescence imaging camera is controlled to take a long exposure once to obtain a single original fluorescence image, which serves as the basis data for subsequent three-dimensional fluorescence imaging. The fluorescence imaging compensation module is used to compensate for the fluorescence intensity in the three-dimensional fluorescence information based on the three-dimensional point cloud, the original fluorescence image and the light attenuation coefficient of the water body, calculate the real fluorescence information, generate the compensated fluorescence image and paste it onto the three-dimensional point cloud. The reflectivity analysis module is used to calculate the surface reflectivity information of the object under test based on calibration parameters and the measured light attenuation coefficient of the water body, and then apply it as a texture map to the 3D point cloud of the object under test calculated after calibration to generate a 3D reflectivity image.

[0005] Preferably, the system includes an external support frame as an overall support and protection structure, used to house and fix the projection system and the detection camera, ensuring their stable operation in complex environments; the projection system and the detection camera are detachably installed inside the external support frame, with the projection system installed at an angle and the detection camera installed vertically, the optical axis of the detection camera forming a certain angle with the optical axis of the projection system, and the optical axes of the projection system and the detection camera intersecting in the target measurement area to form a stereoscopic vision measurement structure.

[0006] Preferably, the projection system includes a projector housing, inside which a projector mounting plate is detachably installed, and a projector is fixedly mounted on the surface of the projector mounting plate. The preferred model of the projector is SLM4710C, and the output end of the projector is detachably fitted with a lens with a narrowband filter.

[0007] Preferably, the detection camera includes a camera housing, inside which a camera mounting plate is detachably installed. A front lens is detachably installed at the end of the camera housing near the object being measured to receive the initial light beam. A collimating lens group is detachably installed at the output end of the front lens. A dichroic mirror is threadedly connected to the output end of the collimating lens group. The dichroic mirror is used to split light of different wavelengths into a transmission path and a reflection path, realizing dual-channel separation of fluorescence imaging and three-dimensional imaging. The dichroic mirror is detachably installed on the surface of the camera mounting plate. A three-dimensional imaging camera is detachably installed at the output end of the transmission path of the dichroic mirror. A right-angle reflector is detachably installed at the output end of the reflection path of the dichroic mirror to bend the light path by 90°. A fluorescence imaging camera is detachably installed at the output end of the light path after the right-angle reflector bends the light path.

[0008] Preferably, the camera calibration module specifically includes: Using Zhang Zhengyou's calibration method, multiple images of a planar calibration plate with known feature point distributions were captured from different directions using a single camera. Based on the planar calibration plate images, the internal parameters of the three-dimensional imaging camera and the fluorescence imaging camera in the detection camera were calculated, including focal length, principal point coordinates and lens distortion coefficients. An independent imaging geometric model was established for each camera. Furthermore, the projection system is calibrated using a projector inverse camera model: the projector is regarded as a virtual camera, and a specific coded stripe pattern is projected onto the calibration board. The projected pattern is then captured by a 3D imaging camera and a fluorescence imaging camera. Based on the known intrinsic parameters of the two cameras, the imaging model of the projector is inversely calculated, and the internal parameters of the projector, including the equivalent focal length, principal point position, and projection distortion parameters, are calculated, thereby completing the intrinsic parameter calibration of the projection system. Based on the intrinsic parameter calibration of each camera and projection system, joint extrinsic parameter calibration of the probe camera and projection system is performed. Without re-acquiring images, the calibration board images obtained during the projection system intrinsic parameter calibration are directly used. Based on common feature points simultaneously observed by the 3D imaging camera and the fluorescence imaging camera, and the already calculated fringe phase, pixel-level correspondences are established between the fields of view of the two cameras and between the camera and the virtual camera model of the projector. On this basis, using the 3D imaging camera coordinate system as a unified reference coordinate system, the extrinsic parameters of the projection system relative to the probe camera are solved to obtain its relative position and orientation, i.e., the rotation matrix and translation matrix. All camera intrinsic parameters, camera extrinsic parameters, and relative pose parameters of the projection system and the external system obtained from calibration are systematically integrated to form a unified world coordinate system. The accuracy of the entire parameter set is verified by the reprojection error index to ensure that the system can perform 3D reconstruction based on the entire parameter set.

[0009] Preferably, the projection end flat field calibration module specifically includes: In an air environment, the projection system projects a pure white image onto a standard reflector with known and uniform reflectivity. The image is captured by a 3D imaging camera, and its grayscale distribution is analyzed. Then, by iteratively adjusting the local grayscale values ​​of the projector's output image, the grayscale values ​​of the image received by the 3D imaging camera within its field of view are made uniform. The projection grayscale compensation map determined by this calibration is recorded as the initial reference state of the system. After the system of the present invention is deployed to the target underwater water area, the light attenuation coefficient of the water body in the working band is measured using a water body light attenuation coefficient measuring instrument, and the subsequent underwater projection pattern is flattened and calibrated in conjunction with the projected grayscale compensation map.

[0010] Preferably, the fluorescence acquisition module specifically includes: Based on the flat-field calibration, the projection system projects six multi-frequency sinusoidal fringe images onto the object under test, including three low-frequency sinusoidal fringes and three high-frequency sinusoidal fringes. The three-dimensional imaging camera in the detection camera is exposed six times simultaneously to capture the fringe deformation image completely. At the same time, the fluorescence imaging camera performs one long exposure during the entire projection sequence to acquire one image as the original fluorescence image. Based on six stripe deformation images captured by a 3D imaging camera, the phase is solved using a multi-frequency heterodyne method. The wrapped phase is unfolded into an absolute phase. Then, combined with pre-calibrated intrinsic and extrinsic parameters, a mapping relationship between the absolute phase and the 3D spatial coordinates is established, and the 3D point cloud of the measured object is calculated. The original fluorescence image acquired by the long exposure of the fluorescence imaging camera, the three-dimensional point cloud of the object under test, and the light attenuation coefficient of the target water body are input into the fluorescence imaging compensation module to compensate the original fluorescence image, thereby obtaining the compensated fluorescence image. The compensated fluorescence image is then used as a texture map and mapped onto the three-dimensional point cloud to obtain three-dimensional fluorescence information.

[0011] Preferably, the fluorescence imaging compensation module specifically includes: The original fluorescence image acquired by the fluorescence imaging camera, the three-dimensional point cloud of the object under test, and the light attenuation coefficient of the target water body obtained through field measurement are extracted to form the basic dataset for fluorescence imaging compensation. Based on the three-dimensional point cloud of the object under test, the spatial position of each pixel is analyzed to obtain the distance information of the target surface, and the fluorescence intensity of each pixel is extracted from the original fluorescence image. By combining the measured water light attenuation coefficient, the distance to the target object, and the fluorescence intensity, the fluorescence intensity is compensated to obtain the true fluorescence intensity, a compensated fluorescence image is generated, and the compensation result is output to the fluorescence acquisition module for 3D mapping.

[0012] Preferably, the reflectivity analysis module specifically includes: The system integrates three input data, including calibration parameters (including grayscale compensation map) determined during air calibration, water light attenuation coefficient measured in the field by a water light attenuation coefficient measuring instrument, and six underwater stripe images of the object under test and the three-dimensional point cloud reconstructed from them, which together constitute the benchmark dataset for reflectance inversion calculation. Based on the known distance between the projection system and the standard reflector during the air calibration phase, the uniform and known reflectivity of the standard reflector, and the grayscale values ​​of the images captured by the 3D imaging camera, the system response relationship between reflectivity and grayscale is established while ignoring air attenuation. The system constant is determined by combining the grayscale compensation map. At the same time, the water light attenuation coefficient measured in the field is extracted, and the distance information of each point of the target object is obtained using the 3D point cloud. Combined with the grayscale distribution of the projected fringe image, an inversion model of the surface reflectivity of the underwater target object is constructed. Based on the system constant, the water light attenuation coefficient, and the grayscale value of the underwater stripe image, the surface reflectance of the underwater target is solved pixel by pixel, thereby calculating the reflectance of each point in the three-dimensional point cloud. The calculated reflectance is then used as a texture map on the three-dimensional point cloud to generate a three-dimensional reflectance image.

[0013] Secondly, an underwater co-aperture three-dimensional fluorescence-reflectivity imaging method, based on the aforementioned underwater co-aperture three-dimensional fluorescence-reflectivity imaging system, includes the following steps: Step 1: In an air environment, complete the calibration of the intrinsic and extrinsic parameters between the 3D imaging camera, the fluorescence imaging camera, and the projection system, and obtain the projection grayscale compensation map through the projection end flat field calibration; In an aquatic environment, measure the water light attenuation coefficient of the target water area in the field to form the basic parameter set required for subsequent 3D reconstruction, reflectivity calculation, and fluorescence compensation. Step 2: Deploy the system of the present invention to the underwater target area. The projection system sequentially projects six multi-frequency sinusoidal fringe images (including three low-frequency fringes and three high-frequency fringes), and the three-dimensional imaging camera simultaneously acquires six fringe deformation images. At the same time, the fluorescence imaging camera performs a long exposure during the entire projection sequence to obtain a single original fluorescence image. Step 3: Based on the six stripe deformation images acquired, the phase is solved using the multi-frequency heterodyne method. The wrapped phase is unfolded into the absolute phase, and the mapping relationship between the phase and the spatial coordinates is established by combining the pre-calibrated intrinsic and extrinsic parameters, thereby calculating the three-dimensional point cloud of the measured object. Step 4: Using the acquired original fluorescence image, combined with the water body light attenuation coefficient and the spatial distance between each point in the three-dimensional point cloud, the fluorescence intensity is compensated pixel by pixel to obtain real fluorescence information; the compensated fluorescence image is then mapped onto the three-dimensional point cloud as a texture map to generate a three-dimensional fluorescence image. Step 5: Based on the calibration parameters obtained from air calibration, the light attenuation coefficient of the water body, and the underwater stripe image, calculate the equivalent flat field image, and then solve the surface reflectance of the target object pixel by pixel; map the reflectance results onto the three-dimensional point cloud to generate a three-dimensional reflectance image.

[0014] This invention provides an underwater co-aperture three-dimensional fluorescence-reflectivity imaging system and method. It has the following beneficial effects: (i) The present invention adopts a common aperture beam splitting imaging design, which splits the reflected light and fluorescence from the same optical aperture into two paths by a dichroic mirror, which are simultaneously received by a three-dimensional imaging camera and a fluorescence imaging camera, respectively. This effectively avoids the data registration error caused by the difference in viewing angle in traditional multi-path imaging, and realizes the spatial consistency and temporal synchronization of three-dimensional point cloud and fluorescence image, which significantly improves the overall accuracy and usability of imaging data.

[0015] (ii) This invention uses projection end flat field calibration technology to perform grayscale uniformity calibration in air, which effectively suppresses the influence of uneven illumination distribution of the projector on the modulation of projection stripes and fluorescence excitation uniformity, ensuring that high-contrast stripe images and stable fluorescence signals can still be obtained in complex underwater environments, and enhancing the environmental adaptability and imaging robustness of the system.

[0016] (III) This invention uses the multi-frequency heterodyne method to solve the phase. Combined with the pre-calibrated camera and projection system parameters, it can quickly and accurately reconstruct the three-dimensional point cloud of the target object. On this basis, by integrating the flat field calibration parameters, water light attenuation coefficient and three-dimensional distance information, the true reflectivity of the object surface is calculated pixel by pixel and mapped onto the three-dimensional point cloud to form a three-dimensional reflectivity image with quantitative optical properties, providing a reliable basis for material identification and surface characteristic analysis. Attached Figure Description

[0017] Figure 1 This is a schematic diagram illustrating the workflow of an underwater co-aperture three-dimensional fluorescence-reflectivity imaging system and method according to the present invention; Figure 2 This is a schematic diagram of the overall appearance and structure of the system of the present invention; Figure 3 This is a schematic diagram of the projection system and detection camera of the present invention; Figure 4 This is a schematic diagram of the internal structure of the projection system and the detection camera of the present invention; Figure 5 This is a schematic diagram of the internal structure of the detection camera of the present invention.

[0018] In the diagram: 1. External support frame of the system; 2. Projection system; 21. Projector housing; 22. Projector; 23. Lens with narrowband filter; 3. Detection camera; 31. Camera housing; 32. Front lens; 33. Collimating lens group; 34. Dichroic mirror; 35. 3D imaging camera; 36. Right-angle mirror; 37. Fluorescence imaging camera. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] Example 1, please refer to Figures 1 to 5 This invention provides a technical solution: an underwater co-aperture three-dimensional fluorescence-reflectivity imaging system, comprising: The external support 1 serves as the overall support and protective structure, housing and fixing the projection system 2 and the detection camera 3 to ensure their stable operation in complex environments. The projection system 2 and the detection camera 3 are detachably installed inside the external support 1. The projection system 2 is installed at an angle, and the detection camera 3 is installed vertically. The optical axis of the detection camera 3 is at a certain angle to the optical axis of the projection system 2. The optical axes of the projection system 2 and the detection camera 3 intersect in the target measurement area, forming a stereoscopic vision measurement structure. The camera housing of the detection camera 3 is equipped with a common aperture beam-splitting imaging component, which is used to receive reflected light and fluorescence from the object under test. The light path from the same optical aperture is divided into a three-dimensional imaging camera light path and a fluorescence imaging camera light path by a dichroic mirror. The three-dimensional morphology and fluorescence information are acquired simultaneously under the same optical aperture, avoiding multi-path registration problems and improving imaging efficiency and data consistency. The projection system 2 includes a projector housing 21. A projector mounting plate is detachably installed inside the projector housing 21. A projector 22 is fixedly mounted on the surface of the projector mounting plate. The preferred model of the projector 22 is SLM4710C. A lens 23 with a narrowband filter is detachably installed at the output end of the projector 22. The projector housing 21 serves as an underwater sealed protective shell for the projector 22 and its related components, ensuring that precision optical components such as the projector are isolated from the high-pressure and corrosive underwater environment, guaranteeing stable and reliable operation of the system underwater for extended periods. The projector mounting plate supports and fixes the projector 22 inside the projector housing 21, ensuring the projector 22's precise and stable installation position within the housing and preventing displacement due to shaking during use. The projector 22 generates and projects sinusoidal fringe images, and the lens 23 with a narrowband filter is installed at the output end of the projector 22 to filter the single-wavelength LED light emitted by the projector 22. The detection camera 3 includes a camera housing 31, inside which a camera mounting plate is detachably installed. A front lens 32 is detachably installed at the end of the camera housing 31 near the object being measured to receive the initial light beam. A collimating lens group 33 is detachably installed at the output end of the front lens 32, immediately following it, for collimating the incident initial light beam. A dichroic mirror 34 is threadedly connected to the output end of the collimating lens group 33. The dichroic mirror 34 is used to split light of different wavelengths into transmitted and reflected light paths, enabling fluorescence imaging and three-dimensional imaging. The system features dual-channel separation, with a dichroic mirror 34 detachably mounted on the surface of the camera mounting plate. A 3D imaging camera 35 is detachably mounted on the output end of the transmission light path of the dichroic mirror 34, directly receiving 3D structured light information for acquiring 3D topography or depth information. A right-angle mirror 36 is detachably mounted on the output end of the reflection light path of the dichroic mirror 34, used to bend the light path by 90°. A fluorescence imaging camera 37 is detachably mounted on the output end of the light path after the bend of the right-angle mirror 36, used to receive signals from the fluorescent sample and complete the acquisition of a two-dimensional fluorescence image.

[0021] The camera calibration module is used to complete the intrinsic parameter calibration of the three-dimensional imaging camera 35 and the fluorescence imaging camera 37 in the detection camera 3, as well as the intrinsic parameter calibration of the projector 22 in the projection system 2, using the Zhang Zhengyou calibration method and the projector inverse camera imaging model. It also uses a multi-view joint calibration method to solve the extrinsic parameters of the fluorescence imaging camera 37 and the projection system 2 relative to the coordinate system of the three-dimensional imaging camera 35, and finally forms a complete calibration parameter set under a unified world coordinate system, providing an accurate geometric basis for subsequent three-dimensional reconstruction.

[0022] The specific work of the camera calibration module is as follows: Using the Zhang Zhengyou calibration method, the 3D imaging camera 35 and the fluorescence imaging camera 37 are driven to capture a series of planar calibration plate images with known feature point distributions from multiple different orientations. Based on these images, the intrinsic parameter matrix of each camera is calculated and solved. The core parameters include the camera's focal length, principal point coordinates, and the radial and tangential distortion coefficients of the lens. After calculation, an independent and accurate imaging geometric model is established for each camera to describe the mapping relationship between two-dimensional pixels and three-dimensional spatial rays. Subsequently, with the calibration plate stationary, the projector 22 in the projection system 2 is treated as a virtual camera. The projector 22 is controlled to project multiple specific coded patterns (preferably Gray code patterns or sine-Gray code combination patterns) onto the calibration plate. The 3D imaging camera 35 and the fluorescence imaging camera 37 acquire the images. Given the intrinsic parameters of the two cameras and the spatial pose of the calibration plate, the equivalent focal length, principal point position, and projection distortion parameters of the projector 22 are calculated based on the inverse camera imaging model, thus completing the intrinsic parameter calibration of the projection system 2.

[0023] Based on the completion of the intrinsic parameter calibration of each camera and projection system 2, without re-acquiring images, the image data obtained during the intrinsic parameter calibration of the projector 22 is directly used. By decoding the projected stripes or coded patterns and performing phase calculations, pixel-level correspondences are established between the fields of view of the 3D imaging camera 35 and the fluorescence imaging camera 37, as well as between the coordinate systems of the two cameras and the virtual camera of the projector 22. Based on these correspondences, a multi-view joint calibration method is adopted, using the coordinate system of the 3D imaging camera 35 as the system's world coordinate system. The external parameters of the fluorescence imaging camera 37 and the projector 22 relative to this world coordinate system are calculated and determined, and the spatial geometric relationships between each imaging unit are accurately expressed in the form of rotation and translation matrices. Finally, the intrinsic parameters of the 3D imaging camera 35, the intrinsic parameters of the fluorescence imaging camera 37, the extrinsic parameters between the two cameras, and the extrinsic parameters with the projector 22 are integrated to form a complete calibration parameter set based on the system's world coordinate system. The reprojection error is used as the key evaluation index. The known world coordinate points are back-projected onto the pixel plane and compared with the coordinates in the actual image to verify the accuracy of the parameter set. This ensures that the system can accurately convert the images from the two cameras and the projected pattern into the 3D point cloud data of the measured object based on this high-precision, integrated parameter set.

[0024] The projection-end flat-field calibration module is used in the projection system 2 to employ a single-wavelength LED light source and a narrow-band filter to ensure the purity of the excitation spectrum. It also uses flat-field calibration technology to calibrate the uniformity of the projected grayscale in air, thereby optimizing the quality of the projected fringes and the uniformity of the excitation, and improving imaging reliability. Specifically, in an air environment, the projection system 2 projects a completely white image onto a standard reflector with known and uniform reflectivity. The 3D imaging camera 35 captures this image and analyzes its grayscale distribution. Then, by iteratively adjusting the local grayscale values ​​of the image output by the projector 22, the grayscale values ​​of the image received within the field of view of the 3D imaging camera 35 are made uniform. The grayscale compensation map of the projector 22 determined by this calibration is recorded as the initial reference state of the system. After deploying the system to the target underwater water area, a water body light attenuation coefficient measuring instrument is used to measure the light attenuation coefficient of the water body in the working wavelength band, and the grayscale compensation map is used to perform flat-field calibration on subsequent underwater projection patterns.

[0025] The specific work of the projection end flat field calibration module is as follows: In an air environment, an initial reference calibration is performed, in which the projection system 2 projects a pure white image onto a standard reflective plate with known and uniform reflectivity. The three-dimensional imaging camera 35 in the detection camera 3 simultaneously captures this image and performs quantitative analysis on the grayscale distribution within its entire field of view. Based on the analysis results, the local grayscale values ​​of different areas in the output image of the projector 22 are dynamically adjusted through an iterative algorithm to compensate for the differences in light intensity distribution caused by inherent system factors such as uneven brightness of the projector itself and lens vignetting. When the grayscale values ​​of each area within the field of view of the three-dimensional imaging camera 35 reach a high degree of uniformity, the calibration process ends, and the grayscale compensation image of the projector 22 determined in this calibration is recorded as the initial reference state of the entire system.

[0026] When the system of this invention is deployed to the target underwater water area, the light attenuation coefficient of the current water body in the working band is measured in the field using a water body light attenuation coefficient measuring instrument to quantify the absorption and scattering effect of the water body on light. Combined with the aforementioned grayscale compensation map, the underwater projection stripe pattern is flat-field calibrated to maintain the uniformity of the projection illuminance distribution as much as possible in the underwater environment, thereby providing stable and consistent light field conditions for three-dimensional reconstruction, reflectivity inversion and fluorescence excitation.

[0027] The fluorescence acquisition module is used to control the fluorescence imaging camera 37 to perform a long exposure during the entire projection sequence while the projection system 2 projects six multi-frequency sinusoidal fringe images, and to integrate and acquire a single original fluorescence image as input data for subsequent fluorescence compensation and three-dimensional fluorescence imaging.

[0028] The fluorescence imaging compensation module is used to compensate for the fluorescence intensity based on the original fluorescence image, the three-dimensional point cloud of the object under test, and the measured light attenuation coefficient of the water body. It calculates the true fluorescence information of the target surface, generates a compensated fluorescence image, and applies the compensated fluorescence image as a texture map onto the three-dimensional point cloud to obtain a three-dimensional fluorescence image.

[0029] The reflectivity analysis module is used to calculate the reflectivity information of the surface of the object under test based on the calibration parameters obtained from air calibration and the water light attenuation coefficient measured in the target water area, combined with the grayscale distribution of the underwater stripe image. The calculated reflectivity is then used as a texture map on the 3D point cloud of the object under test calculated after calibration to generate a 3D reflectivity image.

[0030] Example 2, as Figure 1 As shown in Example 1, the present invention further explains the specific working process of the synchronous acquisition module and the fluorescence acquisition module.

[0031] Based on the flat-field calibration, the projection system 2 sequentially projects six multi-frequency sinusoidal fringe images onto the object under test, including three low-frequency sinusoidal fringes and three high-frequency sinusoidal fringes. The 3D imaging camera 35 in the detection camera 3 simultaneously performs six exposures, capturing each fringe deformation image modulated by the object's surface topography, ensuring complete recording of phase modulation information under different projected fringe patterns. Based on the six fringe deformation images acquired by the 3D imaging camera 35, the phase is deconstructed using a multi-frequency heterodyne method. The wrapped phase is obtained from each frequency fringe image through demodulation. Through stepwise matching and heterodyne operation of high and low frequency phase information, the wrapped phase is unfolded into an absolute phase distribution with global consistency. The obtained absolute phase field is combined with the pre-calibrated intrinsic and extrinsic parameters of the system to establish a precise mapping relationship between the absolute phase value and the 3D spatial coordinates. The 3D coordinates of each point on the surface of the object under test in the system's world coordinate system are calculated pixel by pixel, generating a high-precision 3D point cloud of the object under test.

[0032] In the above-mentioned stripe projection process, six multi-frequency sinusoidal stripe images are sequentially illuminated on the same target in time. The three-step phase-shifting stripes compensate each other in intensity, and the total light intensity after their superposition is equivalent to the superposition of multiple full-white images. Thus, while ensuring the stripe modulation, it provides sufficient and relatively uniform excitation illumination for fluorescent substances, which is beneficial to improving the excitation intensity and signal-to-noise ratio of fluorescent signals under single long exposure conditions.

[0033] The specific operation of the fluorescence acquisition module is as follows: During the projection of the six multi-frequency sinusoidal fringe images by the projection system 2, the fluorescence imaging camera 37 performs a long exposure operation within the entire projection sequence, obtaining a complete original fluorescence image through single integration acquisition, thus achieving synchronous acquisition of three-dimensional morphology data and fluorescence information in time. The original fluorescence image, along with the three-dimensional point cloud of the object under test and the measured light attenuation coefficient of the water body, is input into the fluorescence imaging compensation module to compensate for distance and water body attenuation, generating a compensated fluorescence image. The compensated fluorescence image is mapped onto the reconstructed three-dimensional point cloud as surface texture information. Under the pre-calibrated geometric constraints, accurate registration and mapping between two-dimensional fluorescence pixels and three-dimensional point cloud vertices are achieved, ultimately obtaining a three-dimensional fluorescence imaging result with a true fluorescence distribution.

[0034] Furthermore, to illustrate the complementarity in intensity of the six multi-frequency sinusoidal fringe images, they were divided into two groups of three-step phase-shifted sinusoidal fringe images: one group consisting of three high-frequency sinusoidal fringes, and the other group consisting of three low-frequency sinusoidal fringes. The phase step size of each group of sinusoidal fringe images is [missing information]. ; For any set of sinusoidal fringe images, the grayscale of its projected pattern can be represented as: ; Adding the gray levels of the three images together yields: ; For the cosine term, we have: ; Therefore: ; In the formula: This represents the grayscale distribution of three sinusoidal fringe images of the projection, with each image having a phase difference of [missing information]. ; This means that after flat-field calibration, the DC component of the background light intensity received by the 3D imaging camera when the projection system projects a full white image can be regarded as the gray-level distribution of the full white image after flat-field calibration, which does not change with phase. The modulation amplitude AC component reflects the contrast of the stripes and is related to the surface reflection characteristics of the object and the projection-camera geometry. The phase information is the wrapped phase to be solved, which is related to the three-dimensional shape of the object's surface; As can be deduced above, the result of superimposing each set of three-step phase-shift sinusoidal fringe images is a uniform image without fringe modulation, which is equivalent to a completely white image. The six multi-frequency sinusoidal fringe images used in this invention can be regarded as two sets of three-step phase-shift sinusoidal fringe images. Therefore, the superposition of the six images is equivalent to the superposition of two completely white images.

[0035] In actual measurements, six sinusoidal fringe images are sequentially projected onto the same target. The fluorescence imaging camera 37 performs a long exposure within the entire fringe projection sequence, integrating the fluorescence signals excited under the two equivalent full-white illumination images over time to obtain a single original fluorescence image. Essentially, within the same exposure time, the projector 22 projects two full-white images onto the target and excites fluorescence. This complementary fringe superposition method improves the luminous flux and signal-to-noise ratio of fluorescence excitation while maintaining the structured light modulation characteristics. The fluorescence image, subsequently corrected by the fluorescence imaging compensation module, is then mapped onto the reconstructed 3D point cloud to form 3D fluorescence information with a true fluorescence distribution.

[0036] The fluorescence imaging compensation module specifically includes: extracting the original fluorescence image acquired by the fluorescence imaging camera 37, the three-dimensional point cloud of the object under test, and the light attenuation coefficient of the target water body obtained through field measurement, forming the basic dataset for fluorescence imaging compensation; based on the three-dimensional point cloud of the object under test, analyzing the position of each point in the system's world coordinate system, obtaining the distance information from each point on the target object's surface to the detection camera, and extracting the fluorescence gray value of the corresponding pixel from the original fluorescence image; combining the measured light attenuation coefficient of the water body, the distance of the target object, and the fluorescence gray value, compensating for the fluorescence intensity, inverting the true fluorescence intensity on the object's surface, and obtaining the true fluorescence information.

[0037] The specific function of the fluorescence imaging compensation module is as follows: Structured light projected by projector 22 excites the target object underwater to produce fluorescence in a specific wavelength band. After being dispersed by dichroic mirror 34, the fluorescence image is acquired by fluorescence imaging camera 37, reflecting the fluorescence distribution characteristics of the measured object's surface. Simultaneously, the 3D point cloud data reconstructed by the synchronous acquisition module and the water light attenuation coefficient of the target water area are used to quantify the absorption and scattering effects of the water on the light signal, together forming the basic dataset for fluorescence imaging compensation. Based on this, the fluorescence imaging compensation module obtains the distances to various points of the target object based on the 3D point cloud. And extract the fluorescence grayscale of the corresponding pixels from the original fluorescence image. Combined with the measured light attenuation coefficient of the water body The fluorescence intensity is compensated and calculated to reflect the true fluorescence intensity of the object surface. The true fluorescence information obtained after compensation basically eliminates the influence of environmental factors such as water body attenuation, and more accurately reflects the fluorescence characteristics of the tested object surface.

[0038] According to Beer's Law, the expression for calculating the true gray value of a fluorescence image is as follows: ; In the formula: This represents the true grayscale value of the fluorescence after compensation, eliminating the influence of water attenuation and reflecting the true fluorescence intensity of the object's surface. It represents the light attenuation coefficient in the fluorescence band in water, describing the degree of attenuation of light per unit distance when it propagates in water; This represents a point on the surface of a target object obtained through 3D reconstruction. The distance to the detection camera is the depth information of each point in the 3D point cloud; This indicates that the grayscale value of the fluorescence image actually acquired by the fluorescence imaging camera 37 is an uncompensated measurement value affected by water body attenuation. As a compensation factor for water attenuation, according to the Beer-Lambert law, the attenuation of light propagating in a medium follows an exponential law; therefore, an exponential function is used to compensate for the fluorescence signal. Based on the compensated true grayscale value... The true fluorescence intensity of the target object's surface can be further calculated, and the compensated fluorescence image can be mapped onto the three-dimensional point cloud as a texture map to form a true three-dimensional fluorescence imaging result.

[0039] The reflectivity analysis module specifically includes: integrating three input data, including: (1) calibration parameters obtained after the system is calibrated in air, used to correct the inherent response of the projection and imaging system and establish a gray-scale-radiation response model; (2) the water light attenuation coefficient measured in the field by the water light attenuation coefficient measuring instrument, used to quantify the absorption and scattering effect of the water on the light signal; (3) six underwater phase-shifted fringe images of the object under test collected by the detection camera 3, which serve as the original observation data for subsequent phase calculation and three-dimensional reconstruction, together constituting the benchmark dataset for reflectivity inversion calculation.

[0040] During the data processing phase, the module first establishes a reference baseline based on the air calibration phase: utilizing the known distance between the system and the standard reflector. and the uniform and known reflectivity of a standard reflector Under ideal conditions where air attenuation is ignored, the grayscale values ​​of reference images for 35 pairs of standard reflectors of a 3D imaging camera are obtained. This establishes a mapping relationship between image grayscale and system radiometric response.

[0041] Subsequently, the measured light attenuation coefficient of the water body was extracted for the actual underwater measurement scenario. The phase calculation and 3D reconstruction were performed on six underwater phase-shifted fringe images using the multi-frequency heterodyne method to obtain the distances from each point on the target object's surface to the detection camera. The three high-frequency stripes and three low-frequency stripes are then superimposed pixel by pixel based on the principle of phase-shift complementarity to construct an underwater grayscale image equivalent to "full white illumination" under structured light modulation. .

[0042] Based on this, using system calibration parameters, water light attenuation coefficient, and distance information provided by 3D point cloud, the equivalent full-white grayscale image is analyzed. Attenuation compensation is performed to convert the radiance to the equivalent radiance at each point during underwater imaging. This radiance is then combined with the reference radiance obtained during the air calibration phase to construct an equation for the reflectance of the target surface. By solving this equation, the true reflectance of the underwater target surface is retrieved pixel by pixel. The calculated reflectance data is then used as attribute information and precisely mapped onto the generated 3D point cloud, thus forming a 3D reflectance imaging result that intuitively and quantitatively reflects the optical properties of the object's surface.

[0043] The mathematical expression for the surface reflectivity of an underwater target is as follows: ; ; Where the constant Determined by both projection-imaging geometry and optical system characteristics, it is a scaling factor related to the system structure. The constant is eliminated by the above equation. ,have to: ; In the formula: The surface reflectivity of the underwater target; The distance between the probe camera and the standard reflector during calibration is a known and fixed calibration distance; The reflectivity of a standard reflector is a known, uniform, and calibrated reflectivity value. This represents the grayscale value of the image acquired by the camera during the calibration phase, reflecting the ideal imaging result under conditions of air, known distance, and reflectivity. This represents the light attenuation coefficient of the reflected light band in the water body, which is obtained by actual measurement using a water body light attenuation coefficient measuring instrument. This represents a point on the surface of a target object obtained through 3D reconstruction. Distance to the detection camera; This represents the equivalent full-white grayscale value obtained by superimposing six stripe images during underwater imaging. To ensure consistency with the light intensity during the air calibration stage, the grayscale of each individual image needs to be averaged; therefore, it needs to be divided by 6 in the calculation. (Attenuation term) The coefficient 2 in the figure represents the round-trip optical path (projection optical path + imaging optical path).

[0044] Using the above model, given the standard reflector parameters and the optical properties of the water body, it is possible to determine the model based on the calibrated grayscale. With underwater observation grayscale Surface reflectivity of underwater targets Perform pixel-by-pixel accurate inversion.

[0045] It should be noted that in this specification, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0046] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. An underwater co-aperture three-dimensional fluorescence-reflectivity imaging system, characterized in that, include: The projection system (2) and the detection camera (3) are provided in the detection camera (3), which is equipped with a common aperture beam splitting imaging component to receive reflected light and fluorescence from the object under test, and divides the light path from the same optical aperture into a three-dimensional imaging camera light path and a fluorescence imaging camera light path through a dichroic mirror, and simultaneously collects three-dimensional morphology and fluorescence information under the same optical aperture. The camera calibration module is used to calibrate the intrinsic parameters of the three-dimensional imaging camera and the fluorescence imaging camera in the detection camera (3) using the Zhang Zhengyou calibration method, to calibrate the intrinsic parameters of the projection system (2) using the inverse camera model of the projector, and to obtain the extrinsic parameters between the three-dimensional imaging camera (35), the fluorescence imaging camera (37) and the projection system (2) through multi-view joint calibration. The projection end flat field calibration module is used to use a single-wavelength LED light source and a narrow-band filter in the projection system (2), and to calibrate the gray uniformity of the projection in the air and obtain calibration parameters through flat field calibration technology. The synchronous acquisition module is used to project multi-frequency sinusoidal fringes underwater, use the multi-frequency heterodyne method to solve the phase to obtain the absolute phase, and combine the calibration parameters to calculate the three-dimensional point cloud of the measured object. The fluorescence acquisition module is used to control the fluorescence imaging camera (37) to obtain a single fluorescence image in a long exposure when projecting multi-frequency sinusoidal fringes, which serves as the original fluorescence image for subsequent fluorescence compensation and three-dimensional fluorescence imaging; specifically, it includes: Based on the flat field calibration, the projection system (2) projects six multi-frequency sinusoidal fringe images onto the object under test, including three low-frequency sinusoidal fringes and three high-frequency sinusoidal fringes. The three-dimensional imaging camera (35) in the detection camera (3) is exposed six times simultaneously to capture the fringe deformation image completely. At the same time, the fluorescence imaging camera (37) performs a long exposure during the entire projection sequence to collect a fluorescence image as the original fluorescence image for subsequent fluorescence compensation. Based on the six stripe deformation images captured by the three-dimensional imaging camera (35), the phase is solved by the multi-frequency heterodyne method, the wrapped phase is unfolded into the absolute phase, and then combined with the pre-calibrated internal and external parameters, the mapping relationship between the absolute phase and the three-dimensional spatial coordinates is established, and the three-dimensional point cloud of the measured object is calculated. The fluorescence image compensated by the fluorescence imaging compensation module is used as a texture map and then mapped onto the three-dimensional point cloud to obtain three-dimensional fluorescence information. The fluorescence imaging compensation module is used to compensate for the fluorescence intensity in the fluorescence image, calculate the true fluorescence information, and generate a compensated fluorescence image for three-dimensional fluorescence imaging. The reflectivity analysis module is used to calculate the surface reflectivity information of the object under test based on calibration parameters and the measured light attenuation coefficient of the water body, and then map it onto the 3D point cloud of the object under test calculated after calibration to generate a 3D reflectivity image.

2. The underwater co-aperture three-dimensional fluorescence-reflectivity imaging system according to claim 1, characterized in that: The system includes an external support bracket (1), and the projection system (2) and the detection camera (3) are detachably installed inside the external support bracket (1). The projection system (2) is installed at an angle, and the detection camera (3) is installed vertically. The optical axis of the projection system (2) and the optical axis of the detection camera (3) intersect in the target measurement area.

3. The underwater co-aperture three-dimensional fluorescence-reflectivity imaging system according to claim 2, characterized in that: The projection system (2) includes a projector housing (21), inside which a projector mounting plate is detachably installed, and a projector (22) is fixedly installed on the surface of the projector mounting plate. The output end of the projector (22) is detachably fitted with a lens (23) with a narrowband filter.

4. The underwater co-aperture three-dimensional fluorescence-reflectivity imaging system according to claim 2, characterized in that: The detection camera (3) includes a camera housing (31), a camera mounting plate is detachably installed inside the camera housing (31), a front lens (32) is detachably installed at the end of the camera housing (31) near the object being measured to receive the initial beam, a collimating lens group (33) is detachably installed at the output end of the front lens (32), a dichroic mirror (34) is threadedly connected to the output end of the collimating lens group (33), the dichroic mirror (34) is used to split light of different wavelengths into reflected light paths and transmitted light paths, and the dichroic mirror (34) is detachably installed on the surface of the camera mounting plate, a three-dimensional imaging camera (35) is detachably installed on the output end of the projection light path of the dichroic mirror (34), a right-angle mirror (36) is detachably installed on the output end of the reflected light path of the dichroic mirror (34) to bend the light path by 90°, and a fluorescence imaging camera (37) is detachably installed on the output end of the bend light path of the right-angle mirror (36).

5. The underwater co-aperture three-dimensional fluorescence-reflectivity imaging system according to claim 1, characterized in that: The camera calibration module specifically includes: Using Zhang Zhengyou's calibration method, multiple images of a planar calibration plate with known feature point distributions are captured from different directions using a single camera. Based on the planar calibration plate images, the internal parameters of the three-dimensional imaging camera (35) and the fluorescence imaging camera (37) in the detection camera (3) are calculated, and an independent imaging geometric model is established for each camera. The intrinsic parameters of the projection system (2) are calibrated using the inverse camera model of the projector. The projection system (2) projects multiple specific coded images onto the planar calibration plate. The three-dimensional imaging camera (35) and the fluorescence imaging camera (37) capture the unique Gray code encoding of the feature points of the calibration plate and calculate the pixel position corresponding to the projection system (2), thereby calculating the intrinsic parameters of the projection system (2). Based on the completion of the intrinsic parameter calibration of each camera and projection system (2), the joint extrinsic parameter calibration of the detection camera (3) and the projection system (2) is carried out. Using the calibration plate image used to calibrate the projection system (2), the three-dimensional imaging camera (35), the fluorescence imaging camera (37) and the projection system (2) simultaneously contain the common feature corner points of the calibration plate in the same field of view. Under the premise that the intrinsic parameters of the projection system (2) and the intrinsic parameters of the three-dimensional imaging camera (35) and the fluorescence imaging camera (37) are known, the multi-view joint calibration method is adopted. The coordinate system of the three-dimensional imaging camera (35) is used as the origin of the world coordinate system. The extrinsic parameters of the projection system (2) and the fluorescence imaging camera (37) relative to the world coordinate system are solved to obtain the rotation and translation matrices of the two in the unified three-dimensional world coordinate system. All camera intrinsic parameters, camera extrinsic parameters, and projection system (2) obtained from calibration are systematically integrated with external relative pose parameters to form a unified world coordinate system, and the accuracy of the entire parameter set is verified by the reprojection error index.

6. The underwater co-aperture three-dimensional fluorescence-reflectivity imaging system according to claim 1, characterized in that: The projection end flat field calibration module specifically includes: In an air environment, the projection system (2) projects a pure white image onto a standard reflector with known reflectivity and uniform reflection. By iteratively adjusting the local grayscale value of the output image of the projection system (2), the grayscale value of the image received in the field of view of the three-dimensional imaging camera is made uniform and consistent. The grayscale compensation map determined by this calibration is recorded as the initial reference state of the system. After the system is deployed to the target underwater water area, the light attenuation coefficient of the water body in the working band is measured using a water body light attenuation coefficient measuring instrument, and the subsequent projection pattern is flattened and calibrated in combination with the grayscale compensation map.

7. The underwater co-aperture three-dimensional fluorescence-reflectivity imaging system according to claim 1, characterized in that: The fluorescence imaging compensation module specifically includes: The original fluorescence image acquired by the fluorescence imaging camera (37), the three-dimensional point cloud of the object under test, and the light attenuation coefficient of the target water body obtained by field measurement are extracted to form the basic dataset for fluorescence imaging compensation. Based on the three-dimensional point cloud of the object under test, the spatial position of each pixel is analyzed to obtain the distance information of the target surface, and the fluorescence intensity of each pixel is extracted from the original fluorescence image captured by the fluorescence imaging camera (37). By combining the measured light attenuation coefficient of the water body, the distance of the target object, and the fluorescence intensity, the fluorescence intensity is compensated to obtain the true fluorescence intensity, form true fluorescence information, and the compensated fluorescence image is output to the fluorescence acquisition module for three-dimensional mapping.

8. The underwater co-aperture three-dimensional fluorescence-reflectivity imaging system according to claim 1, characterized in that: The reflectance analysis module specifically includes: The system integrates three input data, including the calibration parameters determined during air calibration, the water light attenuation coefficient measured in the field by the water light attenuation coefficient measuring instrument, and six underwater stripe images of the object under test and the three-dimensional point cloud reconstructed from them, which together constitute the benchmark dataset for reflectance inversion calculation. Based on the known distance between the projection system (2) and the standard reflector during the air calibration stage, the uniform and known reflectivity of the standard reflector, and the grayscale values ​​of the images captured by the three-dimensional imaging camera, the system response relationship between reflectivity and grayscale is established under the premise of ignoring air attenuation, and the system constant is determined by combining the grayscale compensation map; at the same time, the water light attenuation coefficient measured in the field is extracted, and the distance information of each point of the target object is obtained by using the three-dimensional point cloud, and the inversion model of the surface reflectivity of the underwater target object is constructed by combining the grayscale distribution of the projection stripe image; Based on the system constant, the water light attenuation coefficient, and the grayscale value of the stripe deformation image, the surface reflectance of the underwater target is solved pixel by pixel, thereby calculating the reflectance of each point in the three-dimensional point cloud. The calculated reflectance is then used as a texture map on the three-dimensional point cloud to generate a three-dimensional reflectance image.

9. An underwater co-aperture three-dimensional fluorescence-reflectance imaging method, implemented based on the underwater co-aperture three-dimensional fluorescence-reflectance imaging system according to any one of claims 1-8, characterized in that, Includes the following steps: Step 1: In an air environment, calibrate the camera-projection system and generate a grayscale compensation map; in an aquatic environment, measure the light attenuation coefficient of the target water body. Step 2: Deploy the common aperture spectrophotometer system to the underwater target area. The projection system (2) projects six multi-frequency sinusoidal fringe images in sequence, including three low-frequency sinusoidal fringes and three high-frequency sinusoidal fringes. The three-dimensional imaging camera (35) in the detection camera (3) is exposed six times simultaneously to acquire six fringe deformation images. At the same time, the fluorescence imaging camera (37) performs a long exposure during the entire projection sequence to acquire a single original fluorescence image. Step 3: Based on the six stripe deformation images, the phase is solved using the multi-frequency heterodyne method. The wrapped phase is unfolded into the absolute phase. Then, combined with the pre-calibrated intrinsic and extrinsic parameters, the mapping relationship between the absolute phase and the three-dimensional spatial coordinates is established, and the three-dimensional point cloud of the measured object is calculated. Step 4: Based on the three-dimensional point cloud, obtain the distance information of each point of the target object, and combine it with the measured water light attenuation coefficient to compensate the fluorescence intensity of each pixel in the original fluorescence image to obtain the real fluorescence information. Then, use the compensated fluorescence image as a texture map on the three-dimensional point cloud to generate a three-dimensional fluorescence image. Step 5: Based on the grayscale compensation map, the system response parameters obtained from air calibration, the light attenuation coefficient of the water body, and the grayscale distribution of the stripe deformation image, solve for the surface reflectivity information of the object under test, and use the calculated reflectivity as a texture map on the three-dimensional point cloud to generate a three-dimensional reflectivity image.

Images