Dynamic light scattering imaging recovery and displacement prediction method, system and apparatus

Abstract

The application is suitable for the technical field of computer vision, and provides a dynamic optical scattering imaging recovery and displacement prediction method, system and device. The dynamic optical scattering imaging recovery and displacement prediction method comprises: acquiring a second speckle image; extracting motion features of the second speckle image from a time sequence forward sequence and a time sequence reverse sequence respectively through a bidirectional time sequence-phase recurrent neural network, fusing to obtain a third speckle image; and performing frequency domain enhancement on the third speckle image by using phase information in a Fourier domain, and combining a frequency domain attention mechanism to obtain a fourth speckle image. The application realizes robust suppression of speckle noise and cross-frame consistency recovery of texture details in a complex dynamic scene through bidirectional time sequence modeling, sets a frequency domain enhancement module to improve phase spectrum expression capability, and further enhances the structural consistency of a reconstructed image.

Description

Technical Field

[0001] This invention belongs to the field of computer vision technology, and particularly relates to methods, systems and devices for dynamic optical scattering imaging recovery and displacement prediction. Background Technology

[0002] Scattering imaging technology in static scattering media scenarios reconstructs target information by analyzing the physical characteristics of light propagation in non-dynamic scattering media (such as frosted glass, uniform fog, etc.). Existing technologies utilize RNN (Recurrent Neural Network) temporal information transfer and phase spectrum enhancement strategies to enable pixel-level motion trajectory prediction in static scattering media scenarios, providing a reliable solution for scattering imaging.

[0003] However, in practical applications, targets are often in dynamic scattering media environments (such as blood flow in biological tissues, turbid water, etc.), where the properties of the medium evolve nonlinearly over time, causing time-varying distortion of the scattering point spread function (PSF), which in turn renders traditional static imaging methods ineffective. In this dynamic scattering process, scattering imaging of dynamic objects faces two major challenges: first, real-time microscopic fluctuations in the medium disrupt the temporal stability of speckle correlation; second, the coupling between target motion and medium changes leads to aliasing of frequency domain features, significantly increasing the difficulty of recovery.

[0004] Therefore, existing technologies still have shortcomings in dynamic optical scattering imaging. Summary of the Invention

[0005] The purpose of this invention is to provide a method for dynamic optical scattering imaging recovery and displacement prediction, aiming to solve the problem of low accuracy in dynamic optical scattering imaging.

[0006] The present invention is implemented as follows: a method for dynamic optical scattering imaging reconstruction and displacement prediction, the method comprising:

[0007] Acquire the second speckle image;

[0008] Motion features of the second speckle image are extracted from the forward and reverse temporal sequences using a bidirectional time-phase recurrent neural network, and then fused to obtain the third speckle image.

[0009] The third speckle image is enhanced in the frequency domain using phase information in the Fourier domain, and a fourth speckle image is obtained by combining the frequency domain attention mechanism.

[0010] Furthermore, the second speckle image is a preprocessed image, and the steps to obtain the second speckle image are as follows:

[0011] Acquire the first speckle image;

[0012] The first speckle image is cropped and downsampled, and the background illumination is estimated by Gaussian filtering to obtain the background image;

[0013] Perform logarithmic transformations on the first speckle image and the background image respectively, and calculate their difference;

[0014] The linear domain is recovered by exponential transformation and then normalized.

[0015] Enhance image details using contrast-limited adaptive histogram equalization.

[0016] Furthermore, the bidirectional time-phase recurrent neural network includes:

[0017] In the forward-order branch, the first output feature is obtained by modeling the target motion evolution law and the causal propagation characteristics of scattering noise through the hidden state model;

[0018] In the reverse temporal branch, the spatiotemporal correlation between the reverse motion mode and the phase distortion of the medium is captured based on the future hidden state to obtain the second output feature;

[0019] The first output feature and the second output feature are concatenated via channels to generate a fused temporal feature;

[0020] The fused temporal features are used to reconstruct the descattering sequence through a multi-scale residual network to obtain the third speckle image.

[0021] Furthermore, the process of modeling the target motion evolution law and the causal propagation characteristics of scattering noise through hidden states to obtain the first output feature specifically includes:

[0022] Obtain the t-th second speckle image in ascending order, perform downsampling feature extraction on the second speckle image, concatenate the current feature of the t-th second speckle image with the hidden state of the (t-1)-th second speckle image, and obtain the t-th first output feature through residual dense connection and hollow pyramid pooling; where t is a natural number, and the t-th first output feature contains the hidden state of the t-th second speckle image.

[0023] Furthermore, the second output feature obtained based on the spatiotemporal correlation between capturing the inverse motion pattern and the medium phase distortion in the future hidden state specifically includes:

[0024] Obtain the t-th second speckle image in ascending order, perform downsampling feature extraction on the second speckle image, concatenate the current feature of the t-th second speckle image with the future hidden state of the (t+1)-th second speckle image, and obtain the t-th second output feature through residual dense connection and hollow pyramid pooling; where t is a natural number, and the t-th second output feature contains the future hidden state of the t-th second speckle image.

[0025] Furthermore, the step of using phase information in the Fourier domain to enhance the third speckle image in the frequency domain, and then combining this with a frequency domain attention mechanism to obtain the fourth speckle image, specifically includes the following steps:

[0026] Apply a Fast Fourier Transform to the third speckle image to separate the real and imaginary parts;

[0027] The real part is compressed and adjusted by convolution, and the imaginary part is extracted with global average pooling to extract global response features. The local phase structure is enhanced by convolution, and the two are then recombine in the frequency domain to form enhanced complex spectral features.

[0028] The complex spectral features are mapped back to the spatial domain by inverse Fourier transform, and then fused with the third speckle image through residual connection. The fourth speckle image is obtained by combining the frequency domain attention mechanism.

[0029] Furthermore, the frequency domain attention mechanism specifically includes the following steps:

[0030] A lightweight convolutional network is used to generate attention weight matrices across the frequency domain.

[0031] Adaptive reconstruction of low-frequency features is achieved through a gating fusion strategy.

[0032] Another objective of this invention is a dynamic optical scattering imaging reconstruction and displacement prediction system, the system comprising:

[0033] The acquisition module is used to acquire the first speckle image;

[0034] The image preprocessing module is used to preprocess the first speckle image using the Retine enhancement algorithm and contrast-limited adaptive histogram equalization to obtain the second speckle image.

[0035] A bidirectional temporal-phase recurrent neural network is used to extract motion features from the second speckle image from the forward temporal sequence and the reverse temporal sequence, respectively, and fuse them to obtain the third speckle image;

[0036] The frequency domain enhancement module is used to enhance the third speckle image in the frequency domain using phase information in the Fourier domain, and then combine it with the frequency domain attention mechanism to obtain the fourth speckle image.

[0037] Furthermore, the bidirectional time-phase recurrent neural network includes:

[0038] The orthogonal RNN unit models the target motion evolution law and the causal propagation characteristics of scattering noise through the hidden state to obtain the first output feature. The specific operation of the orthogonal RNN unit is as follows: acquire the t-th orthogonal second speckle image, perform downsampling feature extraction on the second speckle image, concatenate the current feature of the t-th second speckle image with the hidden state of the (t-1)-th second speckle image, and obtain the t-th first output feature through residual dense connection and hollow pyramid pooling; where t is a natural number, and the t-th first output feature contains the hidden state of the t-th second speckle image.

[0039] The reverse-order RNN unit captures the spatiotemporal correlation between the inverse motion pattern and the phase distortion of the medium based on the future hidden state, and obtains the second output feature. The specific operation of the reverse-order RNN unit is as follows: acquire the t-th second speckle image in the forward order, perform downsampling feature extraction on the second speckle image, concatenate the current feature of the t-th second speckle image with the future hidden state of the (t+1)-th second speckle image, and obtain the t-th second output feature through residual dense connection and hollow pyramid pooling; where t is a natural number, and the t-th second output feature contains the future hidden state of the t-th second speckle image.

[0040] The feature fusion unit concatenates the first output feature and the second output feature via channels to generate a fused temporal feature;

[0041] The feature reconstruction unit reconstructs the descattering sequence through a multi-scale residual network to obtain the third speckle image.

[0042] Furthermore, the frequency domain enhancement module includes the following:

[0043] The Fast Fourier Transform (FFT) unit applies a Fast Fourier Transform to the third speckle image to separate the real and imaginary parts.

[0044] The recombining unit, wherein the real part is compressed and adjusted by convolution, the imaginary part is extracted with global average pooling to extract global response features, the local phase structure is enhanced by convolution, and the two are recombined in the frequency domain to form enhanced complex spectral features;

[0045] The fast inverse Fourier transform unit maps the complex spectral features back to the spatial domain through the fast inverse Fourier transform, and then fuses them with the third speckle image through residual connection.

[0046] The attention mechanism unit combines the frequency domain attention mechanism to obtain the fourth speckle image; the frequency domain attention mechanism specifically includes: generating cross-frequency domain attention weight matrices using a lightweight convolutional network; and achieving adaptive reconstruction of low-frequency features through a gating fusion strategy.

[0047] Another objective of this invention is a dynamic optical scattering imaging restoration and displacement prediction device, which includes a first light source, a beam expander, an aperture, a spatial light modulator, a lens, a CMOS camera, and a computer device. The first light source emits a light beam that passes sequentially through the beam expander, the aperture, the spatial light modulator, and the lens, through a scattering medium, and is received by the CMOS camera. The speckle image data is then transmitted to the computer device, which is used for image data processing.

[0048] The dynamic optical scattering imaging recovery and displacement prediction device further includes a second light source, an optical power probe, and an optical power meter. The second light source is used to emit a light beam, which passes through the scattering medium and is then detected by the optical power probe. The optical power meter is used to record the optical power and calculate the thickness of the scattering medium.

[0049] The present invention provides a dynamic optical scattering imaging restoration and displacement prediction method, the advantages of which are as follows: This embodiment captures the dual evolution law of medium dynamic scattering and target motion through bidirectional temporal modeling, and achieves robust suppression of speckle noise and cross-frame consistency restoration of texture details in complex dynamic scenes through a time-frequency domain collaborative feature enhancement mechanism; a frequency domain enhancement module is set up, the core of which is to explicitly use the phase information in the Fourier domain to perform structural enhancement on the dynamic speckle image. This module focuses on modeling the imaginary part in the frequency domain to improve the phase spectrum expression ability, thereby enhancing the structural consistency of the reconstructed image. Attached Figure Description

[0050] Figure 1 This is a structural diagram of the dynamic optical scattering imaging recovery and displacement prediction device provided in an embodiment of the present invention;

[0051] Figure 2 A flowchart of the dynamic optical scattering imaging restoration and displacement prediction method provided in an embodiment of the present invention;

[0052] Figure 3 This is a comparison image of dynamic medium speckle treatment before and after, provided in an embodiment of the present invention.

[0053] Figure 4 The flow chart for the dynamic medium speckle pretreatment module provided in this embodiment of the invention;

[0054] Figure 5 This is a diagram of the overall structure of BTP-RNN provided in an embodiment of the present invention;

[0055] Figure 6 This is a schematic diagram of a bidirectional RNN unit structure provided in an embodiment of the present invention;

[0056] Figure 7This is a schematic diagram of the frequency domain enhancement module FEB structure provided in an embodiment of the present invention;

[0057] Figure 8 This is an overall structural diagram of the frequency domain attention mechanism provided in an embodiment of the present invention;

[0058] Figure 9 This is a structural diagram of the dynamic optical scattering imaging restoration and displacement prediction system provided in an embodiment of the present invention;

[0059] Figure 10 This is a block diagram of the internal structure of a computer device in one embodiment. Detailed Implementation

[0060] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0061] It is understood that the terms "first," "second," etc., used in this application may be used herein to describe various elements, but unless otherwise specified, these elements are not limited by these terms. These terms are only used to distinguish one element from another. For example, without departing from the scope of this application, a first script may be referred to as a second script, and similarly, a second script may be referred to as a first script.

[0062] In the first embodiment, such as Figure 1 As shown, a dynamic optical scattering imaging recovery and displacement prediction device is proposed. This device is an application environment diagram of the dynamic optical scattering imaging recovery and displacement prediction method provided in the following embodiments of the present invention.

[0063] The dynamic optical scattering imaging restoration and displacement prediction device includes a first light source 1, a beam expander 2, an aperture 3, a spatial light modulator 4, a lens 5, a CMOS camera 6, and a computer device 7. The first light source 1 is used to emit a light beam, which passes sequentially through the beam expander 2, the aperture 3, the spatial light modulator 4, and the lens 5, passes through the scattering medium, and is received by the CMOS camera 6. The speckle image data is transmitted to the computer device 7, which is used for image data processing.

[0064] The dynamic optical scattering imaging recovery and displacement prediction device further includes a second light source 8, an optical power probe 9, and an optical power meter 10. The second light source 8 is used to emit a light beam, which passes through the scattering medium and is then detected by the optical power probe 9. The optical power meter 10 is used to record the optical power and calculate the thickness of the scattering medium.

[0065] In this embodiment, the computer device 7 can be a tablet computer, a laptop computer, or a desktop computer; it can be an independent physical server or terminal; it can be a server cluster consisting of multiple physical servers; or it can be a cloud server providing basic cloud computing services such as cloud servers, cloud databases, cloud storage, and CDN. The connection to the computer device can be wired or wireless.

[0066] In this embodiment, Figure 1 The optical path configuration used to acquire the first speckle image described in the following embodiments is significantly improved compared to existing technologies. The first light source 1 is a red LED (Thorlabs, M625L4) with a wavelength of 625 nm. This incoherent light source can effectively simulate lighting conditions in a real environment. The diverging beam generated by the first light source 1 is first collimated by a beam expander 2 to form parallel light. After stray light is filtered out by an aperture 3, it is incident on a spatial light modulator 4 (PLUTO-2.1, Holoeye, pixel size 8.0 μm), carrying the optical information of the moving target through reflection. The modulated beam is focused by a lens 5 with a focal length of 300 mm and guided into a scattering medium container 11. This container 11 is a water tank made of acrylic material (20 cm × 10 cm × 40 cm, wall thickness 4 mm), filled with 3.53 L of a suspension prepared from 20% intalripetal emulsion and distilled water. To quantify the intensity of the scattering medium, a second light source 8, a 635nm reference laser source (LR-TRL-635, Changchun Raytron Technology), was placed parallel to the optical path. Its transmitted light intensity was received by a photodiode power probe 9, and the backlight power data was recorded by an optical power meter 10 (Thorlabs, PM100D) for optical thickness calculation. The final scattered light field was captured by a high-sensitivity CMOS camera (BFS-U3-123S6C-C, FLIR, 3.45μm pixel size) to obtain the first speckle image. This device configuration ensures complete capture of the speckle features of dynamic targets.

[0067] This embodiment focuses on the imaging challenges in dynamic scattering media scenarios. The core differences between dynamic and static media are reflected in two aspects: First, the optical properties of the medium evolve nonlinearly over time, resulting in a time-varying point spread function (PSF); second, the optical thickness of dynamic media needs to be strictly quantified through real-time measurement, which poses new requirements for optical path design and data acquisition. To address these challenges, this embodiment redesigns the scattering imaging optical path system, focusing on solving three key problems: (1) maintaining stable speckle signal acquisition under dynamic medium disturbances; (2) achieving high-precision measurement of optical thickness; and (3) constructing a dataset adapted to time-varying scattering characteristics. Specifically, first, a dynamic scattering medium speckle acquisition optical path based on an LED light source is built; second, the optical thickness of the scattering medium is quantitatively calculated using a laser reference beam and an optical power meter; finally, dynamic speckle sequences covering different optical thicknesses, target motion modes, and illumination conditions are acquired, and preprocessing operations are performed using the Retinex (image enhancement) algorithm and the CLAHE algorithm to reduce the impact of changes in experimental conditions (such as scattering medium concentration, illumination conditions, and differences in optical systems), helping the model to better adapt to different scattering environments. This device provides a reliable data foundation for verifying the time-varying scattering robustness of subsequent bidirectional temporal-phase recurrent neural network (BTP-RNN) models.

[0068] In the second embodiment, as Figure 2 As shown, a dynamic optical scattering imaging reconstruction and displacement prediction method is proposed. This embodiment mainly applies this method to the above-mentioned... Figure 1 Using a computer device as an example, a dynamic optical scattering imaging reconstruction and displacement prediction method may specifically include the following steps S100~S400:

[0069] Step S100: Obtain the first speckle image.

[0070] In this embodiment, the first speckle image is obtained by the apparatus of the first embodiment, which will not be described in detail here.

[0071] Step S200: The first speckle image is preprocessed using the Retine enhancement algorithm and contrast-limited adaptive histogram equalization to obtain the second speckle image.

[0072] In this embodiment, we found from experiments that the acquired first speckle image has problems such as low brightness and unclear features. To address this, this embodiment designed the following datasets to verify the recovery ability of BTP-RNN under different scattering conditions, different motion states, and different targets:

[0073] Group 1 (Basic Validation): Using MNIST handwritten digits as the target object, the model is horizontally shifted by 16 pixels per frame. This group validates the model's basic reconstruction capability in dynamic media through regular motion patterns.

[0074] The second group (target complexity verification): Based on a fat emulsion concentration of 2.1 mL and horizontal motion parameters of 16 pixels / frame, the target was replaced with FashionMNIST clothing images. By introducing complex features such as clothing texture and accessory details, scattering interference caused by the diversity of target shapes in real-world scenes was simulated to evaluate the model's adaptability to target structural complexity.

[0075] The third group (verification of motion randomness): The MNIST target is reused under the same scattering conditions, but with a random starting point and multi-directional elastic collision motion (random reflection upon encountering the boundary). The robustness of the model under extreme random perturbations is tested by coupling the dynamic fluctuations of the coupling medium with the uncertainty of the target motion.

[0076] The experiment strictly controlled the temporal correlation: considering the temporal correlation characteristics of the dynamic scattering medium, the exposure time of the high-speed camera was set to 1 second / frame, and the refresh rate of the spatial light modulator was strictly synchronized with the shutter to eliminate the interference of correlation on the experiment.

[0077] After completing the acquisition of speckle sequence data through the above settings, it can be found that, compared with speckle in static scattering media, speckle in dynamic scattering media has problems such as low brightness and indistinct features. Figure 3 As shown, this is due to the insufficient penetration of the LED light source in the dynamic scattering medium, which leads to image degradation. When the optical thickness increases, the LED beam undergoes multiple scattering in the fat emulsion solution, resulting in severe attenuation of the ballistic light signal. The acquired speckle sequence exhibits characteristics such as global brightness attenuation, contrast degradation, and loss of high-frequency details.

[0078] To address this issue, step S200 employs an image preprocessing module to preprocess the image, and the processing flow is as follows: Figure 4 As shown, step S200 specifically includes steps S210 to S250:

[0079] Step S210: Obtain the first speckle image;

[0080] Step S220: Crop and downsample the first speckle image, and estimate the background illumination by Gaussian filtering to obtain the background image;

[0081] Step S230: Perform logarithmic transformation on the first speckle image and the background image respectively, and calculate their difference;

[0082] Step S240: Recover the linear domain through exponential transformation and perform normalization.

[0083] Step S250: Enhance image details using contrast-limited adaptive histogram equalization (CLAHE).

[0084] In steps S210 to S250, the image is preprocessed using a dynamic medium speckle preprocessing module. Step S230 removes the effects of uneven illumination. Step S240 enhances local contrast. Step S250 uses CLAHE to further enhance image details, resulting in a clearer enhanced image.

[0085] Step S300: Motion features of the second speckle image are extracted from the forward and reverse temporal sequences respectively using a bidirectional time-phase recurrent neural network, and then fused to obtain the third speckle image.

[0086] In this embodiment, as Figure 5 As shown, this embodiment proposes a speckle reconstruction framework, BTP-RNN, to address the temporal degradation problem caused by the coupling effect between dynamic scattering media and dynamic targets. This scheme captures the dual evolution of dynamic scattering by the medium and target motion through bidirectional temporal modeling. Through a coordinated feature enhancement mechanism in the temporal and frequency domains, robust suppression of speckle noise and cross-frame consistency restoration of texture details in complex dynamic scenes are achieved.

[0087] Step S400: The third speckle image is enhanced in the frequency domain using phase information in the Fourier domain, and a fourth speckle image is obtained by combining the frequency domain attention mechanism.

[0088] In this embodiment, to enhance the model's ability to recover target structures in dynamic scattering medium scenarios, a Frequency Enhancement Block (FEB) is proposed. Its core design lies in explicitly utilizing phase information in the Fourier domain to enhance the structure of dynamic speckle images. This module focuses on modeling the imaginary part in the frequency domain to improve the phase spectrum representation capability, thereby enhancing the structural consistency of the reconstructed image.

[0089] In a specific scheme, such as Figure 5 and Figure 6 As shown, the bidirectional time-phase recurrent neural network in step S300 includes the following structure:

[0090] In the forward-order branch, the first output feature is obtained by modeling the target motion evolution law and the causal propagation characteristics of scattering noise through the hidden state model;

[0091] In the reverse temporal branch, the spatiotemporal correlation between the reverse motion mode and the phase distortion of the medium is captured based on the future hidden state to obtain the second output feature;

[0092] The first output feature and the second output feature are concatenated via channels to generate a fused temporal feature;

[0093] The fused temporal features are used to reconstruct the descattering sequence through a multi-scale residual network to obtain the third speckle image.

[0094] Need to Figure 5 and 6 The parameters are explained as follows. The acquired second speckle image is a dynamic sequence containing several frames, each frame consisting of images S0~S10. n-1 It means that S0→S n-1 For forward chronological order, S n-1 →S0 represents the reverse order of the time sequence, where n is a natural number greater than 1. S t The second speckle image is input at time t, where t takes values ​​of 0, 1, 2, ..., n-1. H t f H represents the hidden state of the t-th second speckle image in the forward temporal sequence. t b Let F be the future hidden state of the t-th second speckle image in reverse and forward order. t f F is the feature map output at time t in the forward time series. t b F represents the feature map output at time t in reverse and forward order. t For F t f and F t b The fused speckle image, i.e., the third speckle image. t This is the descattering sequence, i.e., the third speckle image. RNN unit. t f For the t-th positive sequence RNN unit, the RNN unit... t b Let t be the t-th reverse-order RNN unit.

[0095] like Figure 6 As shown, for the task of reconstructing moving targets in dynamic scattering scenarios, this network uses bidirectional temporal collaborative modeling as its core. It designs symmetrical forward and backward recurrent neural network branches to extract target motion features from the forward and reverse temporal sequences, respectively. These features are then fused using bidirectional RNN units to achieve global motion trajectory modeling. The forward branch captures the causal propagation characteristics of target inertial motion and scattering noise, while the backward branch decodes the spatiotemporal correlation of reverse motion modes (such as abrupt changes and reciprocating motion) and medium phase distortion. The outputs of both are concatenated to generate fused temporal features, which are finally reconstructed using a multi-scale residual network. This process can be expressed by the following formula:

[0096] ;

[0097] Where i replaces t above, representing the i-th time moment or the i-th frame of the image, RNN f For forward-order timing operations, RNN b This is a reverse timing operation.

[0098] The process of the forward-order temporal sequence is as follows: obtain the t-th second speckle image in the forward order, perform downsampling feature extraction on the second speckle image, concatenate the current feature of the t-th second speckle image with the hidden state of the (t-1)-th second speckle image, and obtain the t-th first output feature through residual dense connection and hollow pyramid pooling; where t is a natural number, and the t-th first output feature contains the hidden state of the t-th second speckle image.

[0099] The reverse temporal sequence process is as follows: obtain the t-th second speckle image in the forward sequence, perform downsampling feature extraction on the second speckle image, concatenate the current feature of the t-th second speckle image with the future hidden state of the (t+1)-th second speckle image, and obtain the t-th second output feature through residual dense connection and hollow pyramid pooling; where t is a natural number, and the t-th second output feature contains the future hidden state of the t-th second speckle image.

[0100] To enhance feature robustness, the output features of the bidirectional RNN units are downsampled and then fused through cross-channel concatenation and convolution to generate the current output. This fusion process fully utilizes the historical accumulated information of the forward features and the future trend prediction of the backward features, significantly improving the modeling accuracy for non-stationary moving targets. This design, through joint optimization of bidirectional temporal collaboration and phase spectrum feature enhancement, effectively decouples dynamic scattering noise and target motion features while maintaining phase structure consistency, providing a highly robust temporal representation for sequence reconstruction under complex media interference.

[0101] In a specific implementation, step S400 specifically includes steps S410 to S440:

[0102] Step S410: Apply a fast Fourier transform to the third speckle image to separate the real and imaginary parts;

[0103] In step S420, the real part is compressed and adjusted by convolution, the imaginary part is extracted by global average pooling to extract global response features, the local phase structure is enhanced by convolution, and the two are recombined in the frequency domain to form enhanced complex spectral features.

[0104] Step S430: The complex spectral features are mapped back to the spatial domain by fast inverse Fourier transform, and then fused with the third speckle image through residual connection.

[0105] Step S440: The fourth speckle image is obtained by combining the frequency domain attention mechanism.

[0106] In this scheme, specifically, as follows: Figure 7 As shown, FEB first applies a Fast Fourier Transform (FFT) to the input feature map, mapping it to the frequency domain to separate the real part (representing amplitude information) and the imaginary part (representing phase information). During processing, the real part is compressed and adjusted using a 1×1 convolution, while the imaginary part is first subjected to global average pooling to extract global response features, and then its local phase structure is enhanced by a 3×3 convolution. The two are then recombined in the frequency domain to form an enhanced complex spectral feature. After mapping this spectrum back to the spatial domain using an Inverse Fourier Transform (IFFT), FEB and the original input are fused through residual connections, forming a frequency-domain-dominated attention enhancement mechanism that improves the expressive power of phase-sensitive regions.

[0107] The design of FEB not only achieves explicit modeling and enhancement of the phase spectrum, but also improves the model's ability to capture structural information in dynamic scattering scenes. In the presence of time-varying perturbations or target displacement amplitudes exceeding the range of optical memory effects, FEB can highlight stable phase features in dynamic speckle, effectively improving edge details and structural consistency in image reconstruction, and enhancing the network's generalization ability and robustness in complex scenes. By introducing decoupling operations of frequency domain features, FEB constructs an enhancement channel distinct from traditional spatial convolution paths, enabling the model to more accurately recover target morphology and boundaries, providing a novel and effective frequency domain modeling approach for dynamic speckle imaging.

[0108] In one specific scheme, the frequency domain attention mechanism in step S440 specifically includes the following steps:

[0109] Step S411: Generate attention weight matrices across the frequency domain using a lightweight convolutional network.

[0110] Step S412: Adaptive reconstruction of low-frequency features is achieved through a gating fusion strategy.

[0111] In this embodiment, high-frequency and low-frequency features in an image typically exhibit close coupling and complementarity. Traditional frequency domain decoupling methods often overlook this inherent cooperation, leading to the fragmentation of cross-frequency band information and making it difficult to simultaneously maintain image semantic integrity and detail sensitivity. To address these issues, this embodiment proposes a frequency domain feature attention mechanism, achieving collaborative optimization of cross-frequency domain information flow. Figure 7 and 8As shown, the frequency domain attention mechanism achieves efficient fusion of cross-frequency domain features through dynamic weight learning and feature interaction. Through learnable spatial-frequency domain attention coefficients, the frequency domain attention mechanism can dynamically balance the contribution of high-frequency and low-frequency features, effectively enhancing the sensitivity to detail changes or anomalies while preserving the integrity of the overall semantic structure of the image, and ultimately achieving collaborative optimization of cross-frequency domain features.

[0112] In the third embodiment, as Figure 9 As shown, a dynamic optical scattering imaging reconstruction and displacement prediction system is provided. This system includes several modules and units, capable of executing the steps and sub-steps of dynamic optical scattering imaging reconstruction and displacement prediction described in the second embodiment above. This embodiment does not list all of these steps, but only provides the following example. The dynamic optical scattering imaging reconstruction and displacement prediction system includes:

[0113] The acquisition module is used to acquire the first speckle image;

[0114] The image preprocessing module is used to preprocess the first speckle image using the Retine enhancement algorithm and contrast-limited adaptive histogram equalization to obtain the second speckle image.

[0115] A bidirectional temporal-phase recurrent neural network is used to extract motion features from the second speckle image from the forward temporal sequence and the reverse temporal sequence, respectively, and fuse them to obtain the third speckle image;

[0116] The frequency domain enhancement module is used to enhance the third speckle image in the frequency domain using phase information in the Fourier domain, and then combine it with the frequency domain attention mechanism to obtain the fourth speckle image.

[0117] In this embodiment, the bidirectional time-phase recurrent neural network described above includes:

[0118] The orthogonal RNN unit models the target motion evolution law and the causal propagation characteristics of scattering noise through the hidden state to obtain the first output feature. The specific operation of the orthogonal RNN unit is as follows: acquire the t-th orthogonal second speckle image, perform downsampling feature extraction on the second speckle image, concatenate the current feature of the t-th second speckle image with the hidden state of the (t-1)-th second speckle image, and obtain the t-th first output feature through residual dense connection and hollow pyramid pooling; where t is a natural number, and the t-th first output feature contains the hidden state of the t-th second speckle image.

[0119] The reverse-order RNN unit captures the spatiotemporal correlation between the inverse motion pattern and the phase distortion of the medium based on the future hidden state, and obtains the second output feature. The specific operation of the reverse-order RNN unit is as follows: acquire the t-th second speckle image in the forward order, perform downsampling feature extraction on the second speckle image, concatenate the current feature of the t-th second speckle image with the future hidden state of the (t+1)-th second speckle image, and obtain the t-th second output feature through residual dense connection and hollow pyramid pooling; where t is a natural number, and the t-th second output feature contains the future hidden state of the t-th second speckle image.

[0120] The feature fusion unit concatenates the first output feature and the second output feature via channels to generate a fused temporal feature;

[0121] The feature reconstruction unit reconstructs the descattering sequence through a multi-scale residual network to obtain the third speckle image.

[0122] In this embodiment, the frequency domain enhancement module includes the following:

[0123] The Fast Fourier Transform (FFT) unit applies a Fast Fourier Transform to the third speckle image to separate the real and imaginary parts.

[0124] The recombining unit, wherein the real part is compressed and adjusted by convolution, the imaginary part is extracted with global average pooling to extract global response features, the local phase structure is enhanced by convolution, and the two are recombined in the frequency domain to form enhanced complex spectral features;

[0125] The fast inverse Fourier transform unit maps the complex spectral features back to the spatial domain through the fast inverse Fourier transform, and then fuses them with the third speckle image through residual connection.

[0126] The attention mechanism unit combines the frequency domain attention mechanism to obtain the fourth speckle image; the frequency domain attention mechanism specifically includes: generating cross-frequency domain attention weight matrices using a lightweight convolutional network; and achieving adaptive reconstruction of low-frequency features through a gating fusion strategy.

[0127] In this embodiment, the dynamic optical scattering imaging recovery and displacement prediction system is also used to solve the two problems of real-time fluctuations in the medium at the microscopic level destroying the time stability of speckle correlation and the coupling between target motion and medium change leading to frequency domain feature aliasing, which will not be elaborated here.

[0128] Figure 10 An internal structural diagram of a computer device in one embodiment is shown. Specifically, this computer device may be... Figure 1 Computer equipment in the environment. For example... Figure 10As shown, the computer device includes a processor, memory, network interface, and input devices connected via a system bus. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system and may also store a computer program. When executed by the processor, this computer program enables the processor to implement a dynamic optical scattering imaging reconstruction and displacement prediction method. The internal memory may also store a computer program, which, when executed by the processor, enables the processor to implement the dynamic optical scattering imaging reconstruction and displacement prediction method. The computer device's display screen can be a liquid crystal display (LCD) or an e-ink display. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the computer device's casing, or an external keyboard, touchpad, or mouse.

[0129] Those skilled in the art will understand that Figure 10 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0130] In one embodiment, a computer device is provided, the computer device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to perform the following steps:

[0131] Acquire the second speckle image;

[0132] Motion features of the second speckle image are extracted from the forward and reverse temporal sequences using a bidirectional time-phase recurrent neural network, and then fused to obtain the third speckle image.

[0133] The third speckle image is enhanced in the frequency domain using phase information in the Fourier domain, and a fourth speckle image is obtained by combining the frequency domain attention mechanism.

[0134] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, causes the processor to perform the following steps:

[0135] Acquire the second speckle image;

[0136] Motion features of the second speckle image are extracted from the forward and reverse temporal sequences using a bidirectional time-phase recurrent neural network, and then fused to obtain the third speckle image.

[0137] The third speckle image is enhanced in the frequency domain using phase information in the Fourier domain, and a fourth speckle image is obtained by combining the frequency domain attention mechanism.

[0138] It should be understood that although the steps in the flowcharts of the various embodiments of the present invention are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the various embodiments may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least a portion of the sub-steps or stages of other steps.

[0139] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0140] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0141] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for dynamic optical scattering imaging reconstruction and displacement prediction, characterized in that, The dynamic optical scattering imaging reconstruction and displacement prediction method includes: Acquire the second speckle image; Motion features of the second speckle image are extracted from the forward and reverse temporal sequences using a bidirectional time-phase recurrent neural network, and then fused to obtain the third speckle image. The third speckle image is enhanced in the frequency domain using phase information in the Fourier domain, and a fourth speckle image is obtained by combining the frequency domain attention mechanism. The bidirectional time-phase recurrent neural network includes: In the forward-order branch, the first output feature is obtained by modeling the target motion evolution law and the causal propagation characteristics of scattering noise through the hidden state model; In the reverse temporal branch, the spatiotemporal correlation between the reverse motion mode and the phase distortion of the medium is captured based on the future hidden state to obtain the second output feature; The first output feature and the second output feature are concatenated via channels to generate a fused temporal feature; The fused temporal features are used to reconstruct the descattering sequence through a multi-scale residual network to obtain the third speckle image; The step of using phase information in the Fourier domain to enhance the third speckle image in the frequency domain, and then combining this with a frequency domain attention mechanism to obtain the fourth speckle image, specifically includes the following steps: Apply a Fast Fourier Transform to the third speckle image to separate the real and imaginary parts; The real part is compressed and adjusted by convolution, the imaginary part is extracted with global average pooling to extract global response features, the local phase structure is enhanced by convolution, and the two are recombined in the frequency domain to form the enhanced complex spectral features. The complex spectral features are mapped back to the spatial domain using a fast inverse Fourier transform, and then fused with the third speckle image through a residual connection. The fourth speckle image is obtained by combining the frequency domain attention mechanism.

2. The dynamic optical scattering imaging reconstruction and displacement prediction method according to claim 1, characterized in that, The second speckle image is a preprocessed image. The steps to obtain the second speckle image are as follows: Acquire the first speckle image; The first speckle image is cropped and downsampled, and the background illumination is estimated by Gaussian filtering to obtain the background image; Perform logarithmic transformations on the first speckle image and the background image respectively, and calculate their difference; The linear domain is recovered by exponential transformation and then normalized. Enhance image details using contrast-limited adaptive histogram equalization.

3. The dynamic optical scattering imaging reconstruction and displacement prediction method according to claim 1, characterized in that, The method of modeling the target motion evolution law and the causal propagation characteristics of scattering noise through hidden states to obtain the first output feature specifically includes: Obtain the t-th second speckle image in ascending order, perform downsampling feature extraction on the second speckle image, concatenate the current feature of the t-th second speckle image with the hidden state of the (t-1)-th second speckle image, and obtain the t-th first output feature through residual dense connection and hollow pyramid pooling; where t is a natural number, and the t-th first output feature contains the hidden state of the t-th second speckle image; The second output feature, obtained by capturing the spatiotemporal correlation between the inverse motion pattern and the phase distortion of the medium based on the future hidden state, specifically includes: Obtain the t-th second speckle image in ascending order, perform downsampling feature extraction on the second speckle image, concatenate the current feature of the t-th second speckle image with the future hidden state of the (t+1)-th second speckle image, and obtain the t-th second output feature through residual dense connection and hollow pyramid pooling; wherein, the t-th second output feature contains the future hidden state of the t-th second speckle image.

4. The dynamic optical scattering imaging reconstruction and displacement prediction method according to claim 1, characterized in that, The frequency domain attention mechanism specifically includes the following steps: A lightweight convolutional network is used to generate attention weight matrices across the frequency domain. Adaptive reconstruction of low-frequency features is achieved through a gating fusion strategy.

5. A dynamic optical scattering imaging reconstruction and displacement prediction system, characterized in that, The dynamic optical scattering imaging reconstruction and displacement prediction system includes: The acquisition module is used to acquire the first speckle image; The image preprocessing module is used to preprocess the first speckle image using the Retine enhancement algorithm and contrast-limited adaptive histogram equalization to obtain the second speckle image. A bidirectional temporal-phase recurrent neural network is used to extract motion features from the second speckle image from the forward temporal sequence and the reverse temporal sequence, respectively, and fuse them to obtain the third speckle image; The frequency domain enhancement module is used to enhance the third speckle image in the frequency domain using phase information in the Fourier domain, and combine it with the frequency domain attention mechanism to obtain the fourth speckle image. The bidirectional time-phase recurrent neural network includes: The orthogonal RNN unit models the target motion evolution law and the causal propagation characteristics of scattering noise through the hidden state to obtain the first output feature. The specific operation of the orthogonal RNN unit is as follows: acquire the t-th orthogonal second speckle image, perform downsampling feature extraction on the second speckle image, concatenate the current feature of the t-th second speckle image with the hidden state of the (t-1)-th second speckle image, and obtain the t-th first output feature through residual dense connection and hollow pyramid pooling; where t is a natural number, and the t-th first output feature contains the hidden state of the t-th second speckle image. The reverse-order RNN unit captures the spatiotemporal correlation between the inverse motion pattern and the phase distortion of the medium based on the future hidden state, and obtains the second output feature. The specific operation of the reverse-order RNN unit is as follows: acquire the t-th second speckle image in the forward order, perform downsampling feature extraction on the second speckle image, concatenate the current feature of the t-th second speckle image with the future hidden state of the (t+1)-th second speckle image, and obtain the t-th second output feature through residual dense connection and hollow pyramid pooling; wherein, the t-th second output feature contains the future hidden state of the t-th second speckle image. The feature fusion unit concatenates the first output feature and the second output feature via channels to generate a fused temporal feature; The feature reconstruction unit reconstructs the descattering sequence through a multi-scale residual network to obtain the third speckle image; The frequency domain enhancement module includes the following: The Fast Fourier Transform (FFT) unit applies a Fast Fourier Transform to the third speckle image to separate the real and imaginary parts. The recombining unit, wherein the real part is compressed and adjusted by convolution, the imaginary part is extracted with global average pooling to extract global response features, the local phase structure is enhanced by convolution, and the two are recombined in the frequency domain to form enhanced complex spectral features; The fast inverse Fourier transform unit maps the complex spectral features back to the spatial domain through fast inverse Fourier transform, and fuses them with the third speckle image through residual connection; The attention mechanism unit combines the frequency domain attention mechanism to obtain the fourth speckle image; the frequency domain attention mechanism specifically includes: generating cross-frequency domain attention weight matrices using a lightweight convolutional network; and achieving adaptive reconstruction of low-frequency features through a gating fusion strategy.

6. A dynamic optical scattering imaging reconstruction and displacement prediction device, characterized in that, The dynamic optical scattering imaging restoration and displacement prediction device includes a first light source, a beam expander, an aperture, a spatial light modulator, a lens, a CMOS camera, and a computer device. The first light source is used to emit a light beam, which passes sequentially through the beam expander, the aperture, the spatial light modulator, and the lens, through the scattering medium, and is received by the CMOS camera. The speckle image data is transmitted to the computer device, which is used for image data processing. The computer device executes the dynamic optical scattering imaging restoration and displacement prediction method according to any one of claims 1-4. The dynamic optical scattering imaging recovery and displacement prediction device further includes a second light source, an optical power probe, and an optical power meter. The second light source is used to emit a light beam, which passes through the scattering medium and is then detected by the optical power probe. The optical power meter is used to record the optical power and calculate the thickness of the scattering medium.

Images