An underwater scattering imaging method based on photon time-of-flight

Abstract

A method for underwater scattering imaging based on photon time-of-flight comprises the following steps: Z1: using a scanning confocal volume scattering imaging system to collect data on a turbid underwater environment, and obtaining three-dimensional spatiotemporal measurements containing information on the time-of-flight of photons. Z2: modeling the scattering imaging process, including a forward propagation model in the illumination phase, and a backward propagation model in the measurement phase, which together constitute a complete scattering imaging model. Z3: deriving an analytical form of the scattering imaging model, and obtaining a boundary migration model of the scattering propagation model. Z4: solving the inverse problem of the analytical scattering propagation model. The method proposed in the present invention can effectively utilize the information of scattered photons, while taking into account the special scattering characteristics of turbid water bodies, and can effectively counteract the backward and forward scattering effects brought about by the turbid underwater environment, achieve clear vision, and provide assistance to underwater resource exploration, underwater search and rescue, underwater operations and other fields.

Description

Technical Field

[0001] The present invention relates to the field of computational imaging technology, and in particular to an underwater scattering imaging method based on photon flight time. Background Art

[0002] Time-of-flight (TOF)-based reconstruction of turbid underwater environments is crucial in practical applications. Existing research has separated scattered and ballistic photons in both the temporal and spatial domains, enabling the reconstruction of objects in weakly scattered underwater scenes. However, absorption and strong anisotropic scattering in turbid underwater environments cause the sparse ballistic light to be tightly coupled between forward and backscattered photons, making separation difficult. This significantly limits the imaging capabilities of existing methods.

[0003] It should be noted that the information disclosed in the above background technology section is only used to understand the background of this application, and therefore may include information that does not constitute prior art known to ordinary technicians in this field. Summary of the Invention

[0004] The main purpose of the present invention is to overcome the defects of the above-mentioned background technology and provide an underwater scattering imaging method based on photon flight time.

[0005] To achieve the above object, the present invention adopts the following technical solutions:

[0006] In a first aspect of the present invention, a method for underwater scattering imaging based on photon time of flight comprises the following steps:

[0007] Z1. Use a scanning confocal volume scattering imaging system to collect data from turbid underwater environments and obtain three-dimensional spatiotemporal measurements containing photon time-of-flight information, including two-dimensional spatial information and one-dimensional temporal information.

[0008] Z2. Model the forward imaging process of the scattering imaging process. This includes a forward propagation model in the illumination phase, which describes the propagation of illumination light in the underwater environment and its interaction with objects, and a backward propagation model in the measurement phase, which describes the process of scattered light returning from the object to the detector. Together, these two models constitute a complete scattering imaging model.

[0009] Z3. Derive the analytical form of the scattering imaging model and obtain the boundary migration model of the scattering propagation model;

[0010] Z4. Solve the inverse problem of the analytical form of the boundary migration model, namely the forward imaging model. The inverse problem of the boundary migration model is solved through five steps: frame-by-frame deconvolution, two-dimensional Fourier transform, inverse numerical integration operation, frequency domain interpolation and three-dimensional inverse Fourier transform. Scattering reconstruction is achieved to restore the target object in the turbid underwater scattering environment.

[0011] In a second aspect of the present invention, a computer-readable storage medium stores a computer program, which, when executed by a processor, implements the underwater scattering imaging method based on photon time-of-flight.

[0012] In a third aspect of the present invention, a computer program product includes a computer program, wherein when the computer program is executed by a processor, the method for underwater scattering imaging based on photon time of flight is implemented.

[0013] Compared with the prior art, the present invention has the following beneficial effects:

[0014] The present invention uses a scanning confocal volume scattering imaging system to capture the photon time-of-flight information of the scene in a turbid underwater strong scattering environment, where the ballistic photons of the target object are strongly attenuated and highly coupled with forward and backscattered photons. In combination with the strong anisotropic scattering characteristics of turbid water, an illumination-reception dual-path imaging model is established in the turbid underwater scattering environment. Furthermore, a boundary migration model is derived based on the forward transmission model, and an analytical mapping optical system between scene information and measurement is established. On this basis, a scattering reconstruction algorithm is implemented by solving the inverse problem. The method proposed by the present invention can effectively utilize the information of scattered photons while taking into account the special scattering characteristics of turbid water. It can effectively counteract the backscattering and forward scattering effects brought about by the turbid underwater environment, achieve clear vision, and provide assistance in underwater resource exploration, underwater search and rescue, underwater operations, and other fields.

[0015] Other beneficial effects of the embodiments of the present invention will be further described below. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 This is an overall process for turbid underwater scattering imaging based on photon time of flight in a preferred embodiment of the present invention;

[0017] Figure 2 Schematic diagram of a confocal volume scattering imaging acquisition system for a turbid underwater environment according to a preferred embodiment of the present invention;

[0018] Figure 3 Schematic diagram of an illumination-detection dual-light-path imaging model for turbid underwater environments according to a preferred embodiment of the present invention;

[0019] Figure 4 Schematic diagram of a boundary migration model for a turbid underwater environment according to a preferred embodiment of the present invention;

[0020] Figure 5 This is a flow chart of a turbid water underwater scattering reconstruction algorithm according to a preferred embodiment of the present invention;

[0021] Figure 6This is the reconstruction test result of the preferred embodiment of the present invention. DETAILED DESCRIPTION

[0022] The following is a detailed description of the embodiments of the present invention. It should be emphasized that the following description is only exemplary and is not intended to limit the scope of the present invention and its application.

[0023] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of such features. In the description of the embodiments of the present invention, "plurality" means two or more, unless otherwise specifically defined.

[0024] The present invention proposes an underwater scattering imaging method based on photon flight time, which is a method for measuring photon flight time information in a turbid underwater scattering environment. The method includes several main parts, such as a turbid underwater scattering imaging model of photon flight time, a boundary migration model based on the underwater imaging model, and a scattering reconstruction algorithm for turbid underwater environments.

[0025] An embodiment of the present invention provides an underwater scattering imaging method based on photon time of flight, comprising the following steps:

[0026] Z1. Use a scanning confocal volume scattering imaging system to collect data from turbid underwater environments and obtain three-dimensional spatiotemporal measurements containing photon time-of-flight information, including two-dimensional spatial information and one-dimensional temporal information.

[0027] Z2. Model the forward imaging process of the scattering imaging process. This includes a forward propagation model in the illumination phase, which describes the propagation of illumination light in the underwater environment and its interaction with objects, and a backward propagation model in the measurement phase, which describes the process of scattered light returning from the object to the detector. Together, these two models constitute a complete scattering imaging model.

[0028] Z3. Derive the analytical form of the scattering imaging model and obtain the boundary migration model of the scattering propagation model;

[0029] Z4. Solve the inverse problem of the analytical form of the boundary migration model, namely the forward imaging model. The inverse problem of the boundary migration model is solved through five steps: frame-by-frame deconvolution, two-dimensional Fourier transform, inverse numerical integration operation, frequency domain interpolation and three-dimensional inverse Fourier transform. Scattering reconstruction is achieved to restore the target object in the turbid underwater scattering environment.

[0030] The present invention discloses an underwater scattering imaging method based on photon time-of-flight information, which includes photon time-of-flight data acquisition in an underwater scattering environment, an imaging model containing time-domain information in a turbid underwater environment, and a scattering imaging algorithm. The time-domain information is combined to accurately describe the propagation process in a turbid underwater scattering environment, thereby realizing scattering reconstruction. In this method, a scanning confocal volume scattering imaging system is used to collect the time-space three-dimensional information of the underwater scattering field. An illumination-detection dual-path imaging model is established, and the imaging process in turbid water bodies is modeled into two sections: the illumination light path and the detection light path. The point spread function and the diffusion equation are used to describe the two scattering propagation processes respectively, and on this basis, the boundary migration model is used to obtain the analytical form of the forward imaging model. The turbid underwater scattering imaging algorithm is implemented by solving the inverse problem of the forward imaging model. The method of the present invention can effectively utilize the information of scattered photons, while taking into account the special scattering characteristics of turbid water bodies, and can effectively counteract the backscattering and forward scattering effects brought about by the turbid underwater environment, thereby achieving clear vision.

[0031] In some embodiments, step Z1 specifically includes: using a scanning confocal volume scattering imaging system to collect three-dimensional spatiotemporal measurement values ​​of the scene, under the control of a galvanometer system, using a pulsed laser to scan the underwater scene point by point, and an ultrafast detector to record the time domain response of each scanning point in turn. The time domain responses of all scanning points together constitute a three-dimensional spatiotemporal measurement value including the photon flight time, recording the time domain propagation process of the scattering field.

[0032] In some embodiments, step Z2 specifically includes: modeling the illumination and detection propagation processes of the turbid water underwater scattering imaging process. For the forward propagation process in the illumination stage, taking into account the strong anisotropic propagation characteristics of turbid water bodies, the underwater point spread function is used to model the changes in the spatial distribution of the light field during the scattering propagation process; for the detection light path that is transmitted back to the detector after the interaction between the illumination light and the scene and is captured by the detector, the diffusion equation containing time information is used to model it. The transmission processes at both ends fully describe the scattering imaging process.

[0033] In some embodiments, step Z2 more specifically includes: using a point spread function to model the spatial distribution changes of the light field during the illumination phase to reflect the strong anisotropic propagation characteristics in turbid water; using a diffusion equation containing time information to model the detection light path, describing the reverse transmission process after the interaction between the illumination light and the scene, and how the scattered light is captured by the detector; constructing a light propagation model in turbid water, quantifying the absorption coefficient and scattering coefficient of light in the illumination light path, and the reflected light effect stimulated by the illumination light; implicitly characterizing the scattering propagation process through the diffusion equation, including time information, while taking into account the scattering characteristics of the turbid water body, assuming that it does not change with time and has spatial translation invariance; determining the boundary conditions of the differential equation, setting the value of the scattered light field at a specific spatial position and time point, and the proportional relationship between the spatial distribution of the light intensity and the bidirectional scattering distribution function of the object; performing spatial convolution of the illumination light field distribution in free space with the forward scattering transfer function of the turbid water body to model the influence of the forward scattered light on the spatial distribution of photons; integrating the forward propagation model of the illumination phase and the backward propagation model of the detection phase to form a complete scattering imaging model.

[0034] In some embodiments, step Z3 specifically includes: using a boundary migration model to abstract the scattering propagation process in the forward imaging model, and deriving an analytical expression of the boundary migration model; wherein, the dual-light path imaging model established by Z2 is abstracted as a boundary migration process from the time boundary to the spatial boundary of the scattered light field, and the analytical expression of the boundary migration model is obtained by solving the first-order linear differential equation and the equivalent change in the frequency domain.

[0035] In some embodiments, step Z3 more specifically includes: abstracting the forward imaging model and converting it into a boundary migration model to describe the conversion process from the time boundary to the spatial boundary of the scattered light field; using a first-order linear differential equation to characterize the boundary migration process, and solving the equation to obtain an analytical expression of the scattering propagation; applying Fourier transform technology to transform the spatial domain of the forward model, and then obtaining a generalized solution in the spatial frequency domain; through equivalent transformation and inverse Fourier transform, deriving a generalized expression of the scattered light field in the spatial domain, thereby obtaining a mapping relationship from scene information to measurement values; introducing dispersion relations and given boundary conditions, constructing a conversion from the frequency domain to the spatial domain, and realizing the spatial boundary expression of the scattered light field; comprehensively using mathematical transformations and boundary conditions to form an analytical form of a complete forward imaging model, and deriving the mapping relationship between scene information and measurement values ​​through analytical expressions, providing a mathematical basis for solving the inverse problem.

[0036] In some embodiments, step Z4 specifically includes: frame-by-frame deconvolution: applying a deconvolution operation to each collected time frame data, utilizing the correlation between depth and photon flight time, and processing through a constructed two-dimensional convolution kernel to eliminate the blurring effect in the data; two-dimensional Fourier transform: performing a two-dimensional Fourier transform on the deconvolved data to convert the spatial domain data to the frequency domain in preparation for further inverse operations; numerical integral inverse operation: performing an inverse operation on the frequency domain data using a transfer matrix, which is derived based on a numerical integral relationship to restore the spatial distribution characteristics of the data; frequency domain interpolation: performing interpolation processing in the frequency domain to achieve expansion from two-dimensional to three-dimensional frequency domain to fill data gaps and enhance the integrity and continuity of the reconstructed image; three-dimensional inverse Fourier transform: performing a three-dimensional inverse Fourier transform on the data that has been processed in the frequency domain, converting the data from the frequency domain back to the spatial domain, and ultimately achieving clear reconstruction of the target object in a turbid underwater scattering environment.

[0037] In some embodiments, the scanning confocal volume scattering imaging system includes a pulsed laser, a beam splitter, a galvanometer system, a scattered light collection system, and an ultrafast detector. The laser light emitted by the pulsed laser passes through the beam splitter to form a main illumination beam. The main illumination beam is irradiated onto an underwater target object through the galvanometer system, exciting scattered light. The scattered light returns and is separated into returned light through the beam splitter. The scattered light collection system receives the returned light and guides it to the ultrafast detector. The ultrafast detector records the time-space information of the scattered light and generates a three-dimensional space-time measurement value containing photon flight time information.

[0038] The present invention is based on photon time-of-flight technology and uses a scanning confocal scattering imaging system to collect photon time-of-flight measurements in a turbid underwater environment. It establishes an illumination-detection dual-path imaging model in a turbid underwater environment, and utilizes the analytical form of the forward imaging model to obtain a turbid underwater scattering reconstruction algorithm by solving the inverse problem of the forward imaging model. This effectively restores the target object in the turbid underwater environment and solves the problem of poor scattering reconstruction quality in a strongly scattered turbid underwater environment. The method of the present invention combines the photon time-of-flight information modeling imaging process to fully describe the scattering propagation process of photons and realizes scattering reconstruction by solving the inverse problem. This can effectively utilize the object information contained in the scattered light, breaking through the performance limits of existing methods. The present invention can reconstruct target objects in a turbid underwater scattering environment and can be used in applications such as underwater resource exploration, underwater operation auxiliary vision, and underwater search and rescue.

[0039] Specific embodiments of the present invention are further described below.

[0040] like Figure 1 FIG. 1 is a flow chart of an underwater scattering imaging method based on photon time of flight according to a preferred embodiment of the present invention, comprising the following steps:

[0041] Step Z1: Use Figure 2 The imaging system shown here is used to capture three-dimensional spatiotemporal measurements of turbid underwater environments, including photon time-of-flight information. This imaging system uses a high-power pulsed laser (INNO AMT-532-1W1M) to provide active illumination. The laser light, with a wavelength of 532 nm, a pulse width of 12 ps, a repetition rate of 20 MHz, and an average optical power of 700 mW, passes through a beamsplitter (Thorlabs PBS251) and is then incident on the front surface of the scene by a galvanometer (Thorlabs GVS012), thereby changing its direction and scanning the scene pixel by pixel under the control of a data acquisition device (NI-DAQ USB-6343). Reflected light from the scene follows the same optical path, passing through the galvanometer and beamsplitter, and is focused directly onto a SPAD array (Photon Force PF32) serving as an ultrafast detector by a lens with a focal length of 2.8-12 mm and a maximum f-number of 1.6. Each pixel of the ultrafast detector operates in time-correlated single-photon counting (TCSPC) mode with a temporal resolution of 55 ps. During acquisition, a pixel located at the center of the lens' focal spot is preselected to provide a photon count histogram for each scan point. A time delay device (Micro PhotonDevices PSD-065-A-MOD) is used to convert the synchronization signal output by the laser into a standard transistor-transistor logic (TTL) signal, which is then used as the acquisition trigger signal for the SPAD array. In this setup, the measurement results of all scan points together constitute a three-dimensional spatiotemporal measurement, which contains two-dimensional spatial information (64×64) and one-dimensional temporal information (250 time bins).

[0042] Step Z2: Figure 3 As shown, the present invention establishes an illumination-detection dual-path imaging model for the imaging process of turbid underwater environments. Figure 3 As shown in (a), the receiving optical path is as follows Figure 3 As shown in (b) in the figure. Based on the assumption that the illumination-return light paths of the confocal imaging system are the same, the dual-light-path imaging model can be equivalent to the single-light-path imaging model of the self-luminous object, and the scattering propagation process can be described as:

[0043] Y(x,y,t)=H(O(x,y,z))+W

[0044] Where Y represents the received TOF measurement value, H(·) represents the scattering transfer function of turbid water, which represents the mapping relationship between scene information and measurement signals. O(x, y, z) ∝ BSDF(x, y, z) indicates that the spatial distribution of the light intensity of the equivalent self-luminous scene is proportional to the bidirectional scattering distribution function of the object at the (x, y, z) position in the scene. W represents additive random measurement noise. Due to the complexity of the scene and the scattering propagation process, the transfer function H(·) is often difficult to express explicitly. Therefore, the diffusion equation with time information is used to implicitly represent the dynamic propagation process:

[0045]

[0046] At the same time, the differential equation contains the following boundary conditions:

[0047]

[0048] Where φ(x, y, z, t) represents the scattered light field, c′=c / 2 is the equivalent speed of light, D(x, y, z)=(3(μ a +(1-g)μ s )) -1 represents the diffusion coefficient, where μ a represents the absorption coefficient, μ s represents the scattering coefficient, and g is the mean cosine of the scattering angle. Meanwhile, s(x, y, z, t) represents the illuminated light field, which includes the active illumination source and the reflected light generated by the illumination light. Generally speaking, s(x, y, z, t) depends only on inherent parameters such as the light source and the reflective properties of the reflective surface. However, in turbid water, the spatial distribution of the illuminated light field can be significantly altered by strong forward scattering. To model the effect of forward scattered light on the spatial distribution of photons, the light source term is modeled as:

[0049] s(x, y, z, t) = H F (s0(x, y, z, t))

[0050] Among them, s0(x, y, z, t) is the illumination light field distribution in free space, H F (·) represents the forward scattering transfer function of turbid water. Assuming that the scattering characteristics of turbid water do not change with time and have spatial translation invariance, the transfer function H F (·) can be modeled as the point spread function P of turbid water w Spatial convolution of (x, y, z) and the original light source spatial distribution s0(x, y, z, t):

[0051] s(x, y, z, t) = H F (s0(x, y, z, t))=p w(x, y, z)*s o (x, y, z, t)

[0052] A large amount of work has been done to study the mathematical expression of the point spread function of turbid water. Among them, Du et al., a representative example, derived a simplified form of the steady-state point spread function of turbid water based on the theory of Wells' small-angle approximation, as follows:

[0053]

[0054] Where K(θ0) is a constant related to the average scattering angle θ0, ω refers to the single scattering albedo of the water body, and And θ(x, y, z) refers to the distance and scattering angle of the position (x, y, z) relative to the reference position (x0, y0, z0), while μ c =μ a +(1-g)μ s Represents the total attenuation coefficient. On this basis, the complete implicit modeling of the forward imaging process in turbid underwater environment can be obtained:

[0055]

[0056] Step Z3: Abstract the forward imaging model into a boundary migration model and derive its analytical mathematical form;

[0057] Specifically, Z2 simulates the imaging process in a turbid underwater environment. However, this implicit mapping relationship makes it difficult to solve the inverse problem of reconstructing the target. Therefore, it is necessary to further derive a resolvable mapping relationship between scene information and time-of-flight measurements. Combined with the provided boundary conditions, the forward imaging model can be abstracted as a boundary migration model between the temporal boundary φ(x, y, z, t = 0) and the spatial boundary φ(x, y, z = 0, t), as shown below:

[0058] φ(x,y,z=0,t)=H(φ(x,y,z,t=0))

[0059] First, consider the implicit function H(·), which is constructed by the spatial domain (x, y, z, t)→(k x , k y , k z , t) and solve the corresponding first-order linear non-chiral differential equation to obtain the scattered light field Φ(k x , k y , k z , t) in the spatial frequency domain has the following generalized solution:

[0060]

[0061] The time-independent part of the generalized solution is expressed as Φ′(k x , k y , k z ), where C(k x , k y , k z ) is the space-frequency term in the generalized solution that is independent of the time domain. On this basis, the equivalent transformation of equation (8) is as follows:

[0062]

[0063] By performing an inverse Fourier transform on both sides, we can obtain a generalized expression for the scattered light field φ(x, y, z, t) in the spatial domain, as shown below:

[0064]

[0065] Then, in order to derive the spatial boundary of z = 0, the dispersion relation is introduced And given the boundary condition z=0, we can get:

[0066]

[0067] in, is the spatial frequency domain term in the scattered light field, and the resulting conversion of scene information to measured values ​​is given by:

[0068]

[0069] in, and F xgz (·) denotes two-dimensional inverse Fourier transform and three-dimensional Fourier transform, respectively. Indicates (k x , k y , k z )→(k x , k y , f) frequency domain mapping relationship. The analytical form of the complete forward imaging model is as follows Figure 4 given.

[0070] Step Z4: Implement the turbid water scattering imaging algorithm by solving the inverse problem of the forward imaging model.

[0071] Specifically, if Figure 5 As shown, the reconstruction algorithm will be implemented separately Figure 4 The inverse process of the five steps in the forward imaging model includes frame deconvolution, two-dimensional Fourier transform, inverse numerical integration, frequency domain interpolation, and three-dimensional inverse Fourier transform.

[0072] First, for each time frame φ(x, y, z = 0, t = t1), a deconvolution operation is performed using a two-dimensional convolution kernel ψ constructed based on the point spread function of turbid water. Since prior information about the depth z is required to calculate the distance l(x, y, z) and the spatial scattering angle θ(x, y, z) during the construction of the convolution kernel, an approximate calculation is performed using the correlation between the depth and the photon flight time z1 = c′·t1. Based on this, the two-dimensional spatial deconvolution of each time frame can be solved using the Wiener filter method as follows:

[0073]

[0074] Among them, F(·) and F -1 (·) represent the Fourier transform and inverse transform respectively. On this basis, the spatial domain (x, y)→(k x , k y ) is Fourier transformed and inverted according to the numerical transformation relationship of the frequency domain f integral, which can be expressed as:

[0075]

[0076] Among them, H is the integral term The transfer matrix is ​​derived by numerical integration relation. Next, according to the diffusion relation (k x , k y , k f ) domain to (k x , k y , k z ) domain, which is achieved through frequency domain interpolation as shown below:

[0077]

[0078] Finally, after the spatial domain inverse Fourier transform (k x , k y , k z )→(x, y, z), the time boundary is obtained:

[0079]

[0080] In order to verify the effect of the method proposed in the present invention, the present invention conducted an experimental test in a simulated turbid water environment, using Maalox as a scattering agent to increase the scattering degree of water, and using flat letters and three-dimensional figures with Lambertian surfaces as target objects in the scene. The test results are as follows: Figure 6 shown. Figure 6The reconstruction results of various two-dimensional and three-dimensional targets using time gating, cross-correlation, BMM, and the method of the present invention are shown. It can be seen that for different targets at different depths, the signal generated by the direct time-gating method cannot distinguish the target, and only a slight bright spot can be seen in the target area. This is because in turbid underwater environments, ballistic photons are strongly scattered and absorbed, resulting in a significantly reduced ratio and close coupling with the background noise generated by backscattering. Simply isolating the ballistic photons in the time dimension cannot effectively extract the ballistic photons. Pixel-level cross-correlation uses the system's time-domain response as a prior to improve the contrast of the reconstruction results. However, in strong underwater scattering environments, the sparsity of ballistic photons still only shows a brighter area, making it difficult to discern the target's shape. The BMM algorithm, based on diffusion equation modeling, reverses the propagation process, and the reconstruction results have significantly higher contrast than time-gating and cross-correlation algorithms. Notably, under conditions of strong anisotropic scattering in water, forward-scattered photons and ballistic photons are highly coupled in the time domain due to the similar propagation path lengths and similar energies during forward propagation. The time-gating and cross-correlation methods don't model scattering propagation, and the BMM algorithm doesn't consider the scattering effect of forward propagation during its modeling. As a result, these three methods' reconstruction results include artifacts created by forward scattered light as part of the object, significantly blurring the spatial form of the object in the reconstruction. Compared to these three algorithms, the present invention fully simulates both stages of the illumination-imaging process in turbid water, effectively reconstructing all clearly shaped objects and effectively separating them from background noise, far outperforming other methods.

[0081] To objectively evaluate the effectiveness of the proposed method, we calculated the PSNR and SSIM for all reconstruction results. As shown in Tables 1 and 2, the proposed method achieved the best PSNR and SSIM. When the ballistic light and scattered light are highly coupled, time gating and cross-correlation are ineffective, and the reconstruction result contains a large amount of scattering noise, resulting in a very low PSNR. The BMM algorithm improves the contrast of the reconstruction results, but it still cannot eliminate the interference of forward scattering noise, and thus still has a low PSNR. By simulating the propagation process in turbid waters, the present invention effectively eliminates the interference of forward and backscattering noise, reconstructing the target object more clearly, thus achieving the highest PSNR among all experiments. At the same time, because time gating, cross-correlation, and BMM do not simulate the forward propagation process in an underwater environment, the reconstruction result is affected by the blurring effect caused by forward scattering, making it almost impossible to distinguish the shape and structure of the target object. The present invention can effectively eliminate the interference caused by this scattering noise and reconstruct a clear target object, resulting in a significantly higher structural similarity than the other three algorithms.

[0082] Table 1

[0083]

[0084] Table 2

[0085]

[0086] An embodiment of the present invention further provides a storage medium for storing a computer program, which at least performs the above method when executed.

[0087] An embodiment of the present invention further provides a control device, comprising a processor and a storage medium for storing a computer program; wherein the processor is configured to execute at least the method described above when executing the computer program.

[0088] An embodiment of the present invention further provides a processor, which executes a computer program and at least performs the method described above.

[0089] The storage medium can be implemented by any type of non-volatile storage device, or a combination thereof. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a flash memory, a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD-ROM); the magnetic surface memory can be a magnetic disk memory or a magnetic tape memory. The storage medium described in the embodiments of the present invention is intended to include, but is not limited to, these and any other suitable types of memory.

[0090] In the several embodiments provided by the present invention, it should be understood that the disclosed systems and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0091] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0092] In addition, all functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may be separately used as a unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0093] Those skilled in the art will appreciate that all or part of the steps of the above-mentioned method embodiments may be implemented by hardware associated with program instructions, and the aforementioned program may be stored in a computer-readable storage medium. When the program is executed, the program executes the steps of the above-mentioned method embodiments. The aforementioned storage medium includes various media that can store program codes, such as mobile storage devices, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical disks.

[0094] Alternatively, if the above-mentioned integrated unit of the present invention is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROM, RAM, magnetic disks or optical disks.

[0095] The methods disclosed in the several method embodiments provided by the present invention can be arbitrarily combined without conflict to obtain new method embodiments.

[0096] The features disclosed in several product embodiments provided by the present invention can be arbitrarily combined without conflict to obtain new product embodiments.

[0097] The features disclosed in several method or device embodiments provided by the present invention can be arbitrarily combined without conflict to obtain new method embodiments or device embodiments.

[0098] The above is a further detailed description of the present invention in conjunction with specific preferred embodiments, and the specific implementation of the present invention should not be considered to be limited to these descriptions. Those skilled in the art will recognize that, without departing from the scope of the present invention, several equivalent substitutions or obvious variations can be made, and the performance or use of the same should be considered to fall within the scope of protection of the present invention.

Claims

1. An underwater scattering imaging method based on photon time of flight, characterized in that: The steps include: Z1. Use a scanning confocal volume scattering imaging system to collect data from turbid underwater environments and obtain three-dimensional spatiotemporal measurements containing photon time-of-flight information, including two-dimensional spatial information and one-dimensional temporal information. Z2. Develop a forward imaging model for the scattering imaging process. This includes a forward propagation model during the illumination phase, which describes the propagation of illumination light in the underwater environment and its interaction with the target object, and a backward propagation model during the measurement phase, which describes the process of scattered light returning from the target object to the detector. Together, these two models form a complete scattering imaging model. Z3. Derive the analytical form of the scattering imaging model and obtain the boundary migration model of the scattering propagation model; Z4. Solve the inverse problem of the analytical form of the boundary migration model, namely the forward imaging model. The inverse problem of the boundary migration model is solved through five steps: frame-by-frame deconvolution, two-dimensional Fourier transform, inverse numerical integration operation, frequency domain interpolation and three-dimensional inverse Fourier transform. Scattering reconstruction is achieved to restore the target object in the turbid underwater scattering environment.

2. The underwater scattering imaging method based on photon time of flight according to claim 1, characterized in that: Step Z1 specifically includes: using a scanning confocal volume scattering imaging system to collect three-dimensional spatiotemporal measurements of the scene. Under the control of a galvanometer system, a pulsed laser is used to scan the underwater scene point by point. An ultrafast detector records the time domain response of each scanning point in turn. The time domain responses of all scanning points together constitute a three-dimensional spatiotemporal measurement value including the photon flight time, recording the time domain propagation process of the scattering field.

3. The underwater scattering imaging method based on photon time of flight according to claim 1, characterized in that: Step Z2 specifically includes: modeling the two propagation processes of illumination and detection in the turbid water scattering imaging process. For the forward propagation process in the illumination stage, the underwater point spread function is used to model the changes in the spatial distribution of the light field during the scattering propagation process; for the detection light path that is transmitted back to the detector after the interaction between the illumination light and the scene, a diffusion equation containing time information is used to model it.

4. The underwater scattering imaging method based on photon time of flight according to claim 3, characterized in that: Step Z2 more specifically includes: using the point spread function to model the spatial distribution changes of the light field during the illumination phase to reflect the strong anisotropic propagation characteristics in turbid water; using the diffusion equation containing time information to model the detection light path, describing the reverse transmission process after the interaction between the illumination light and the scene, and how the scattered light is captured by the detector; constructing a light propagation model in turbid water, quantifying the absorption coefficient and scattering coefficient of light in the illumination light path, and the reflected light effect stimulated by the illumination light; implicitly characterizing the scattering propagation process through the diffusion equation, including time information, while taking into account the scattering characteristics of turbid water, assuming that it does not change with time and has spatial translation invariance; determining the boundary conditions of the differential equation, setting the value of the scattered light field at a specific spatial position and time point, and the proportional relationship between the spatial distribution of light intensity and the bidirectional scattering distribution function of the object; performing spatial convolution of the illumination light field distribution in free space with the forward scattering transfer function of the turbid water to model the influence of forward scattered light on the spatial distribution of photons; integrating the forward propagation model of the illumination phase and the backward propagation model of the detection phase to form a complete scattering imaging model.

5. The underwater scattering imaging method based on photon time of flight according to claim 1, characterized in that: Step Z3 specifically includes: using the boundary migration model to abstract the scattering propagation process in the forward imaging model, and deriving the analytical expression of the boundary migration model; among them, the dual-light path imaging model established in Z2 is abstracted as the boundary migration process from the time boundary to the spatial boundary of the scattered light field, and the analytical expression of the boundary migration model is obtained by solving the first-order linear differential equation and the equivalent change in the frequency domain.

6. The underwater scattering imaging method based on photon time of flight according to claim 5, characterized in that: Step Z3 more specifically includes: abstracting the forward imaging model and converting it into a boundary migration model to describe the conversion process from the time boundary to the spatial boundary of the scattered light field; using a first-order linear differential equation to characterize the boundary migration process, and solving the equation to obtain an analytical expression of the scattering propagation; applying Fourier transform technology to transform the spatial domain of the forward model, and then obtaining a generalized solution in the spatial frequency domain; deriving a generalized expression of the scattered light field in the spatial domain through equivalent transformation and inverse Fourier transform, thereby obtaining a mapping relationship from scene information to measurement values; introducing dispersion relations and given boundary conditions, constructing a conversion from the frequency domain to the spatial domain, and realizing the spatial boundary expression of the scattered light field; comprehensively using mathematical transformations and boundary conditions to form a complete analytical form of the forward imaging model, and deriving the mapping relationship between scene information and measurement values ​​through analytical expressions, providing a mathematical basis for solving the inverse problem.

7. The underwater scattering imaging method based on photon time of flight according to claim 1, characterized in that: Step Z4 specifically includes: Frame-by-frame deconvolution: Deconvolution is applied to each acquired time frame, using the correlation between depth and photon flight time to remove blurring effects from the data through a constructed two-dimensional convolution kernel. Two-dimensional Fourier transform: Perform a two-dimensional Fourier transform on the deconvolved data to convert the spatial domain data into the frequency domain in preparation for further inverse operations; Inverse numerical integration: Perform an inverse operation on the frequency domain data using the transfer matrix, which is derived based on the numerical integration relationship, to recover the spatial distribution characteristics of the data. Frequency domain interpolation: Interpolation processing is performed in the frequency domain to achieve expansion from two-dimensional to three-dimensional frequency domain to fill data gaps and enhance the integrity and continuity of the reconstructed image; 3D inverse Fourier transform: Perform a 3D inverse Fourier transform on the frequency-domain processed data, converting the data from the frequency domain back to the spatial domain, ultimately achieving clear reconstruction of the target object in the turbid underwater scattering environment.

8. The underwater scattering imaging method based on photon time of flight according to claim 1, characterized in that: The scanning confocal volume scattering imaging system includes a pulsed laser, a beam splitter, a galvanometer system, a scattered light collection system, and an ultrafast detector. The laser light emitted by the pulsed laser passes through the beam splitter to form a main illumination beam. The main illumination beam is irradiated onto an underwater target object through the galvanometer system, thereby stimulating scattered light. The scattered light returns and is separated into return light through the beam splitter. The scattered light collection system receives the return light and guides it to the ultrafast detector. The ultrafast detector records the time-space information of the scattered light and generates a three-dimensional space-time measurement value including the photon flight time information.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the underwater scattering imaging method based on photon flight time as described in any one of claims 1 to 8 is implemented.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the underwater scattering imaging method based on photon flight time as described in any one of claims 1 to 8 is implemented.

Images