An Adaptive Filtering Method for Frequency Division Multiplexing Ultrafast Imaging Reconstruction

Abstract

This invention discloses an adaptive filtering method for frequency division multiplexing (FDM) ultrafast imaging reconstruction, belonging to the field of computational optical imaging technology. The steps are as follows: A large number of multi-amplitude FDM ultrafast imaging raw images are simulated and generated using a random matrix generation and interpolation method combined with grating modulation principles; unmodulated random complex amplitude images are removed from each raw image to obtain images without zero-order terms; the pix2pixGAN generative adversarial network is improved and trained; the actual captured multi-amplitude FDM images are input into the trained network, outputting multi-amplitude FDM images without zero-order terms, and a Fourier transform is performed to obtain a spectrum without zero-order terms. The spectrum is binarized, and the K-Means++ clustering algorithm is applied to obtain the center coordinates of the positive and negative first-order terms of the spectrum. The single-frame image information corresponding to the center coordinates is extracted using a Butterworth filter, and an inverse Fourier transform is performed to obtain the single-frame FDM image, thus achieving adaptive filtering.

Description

Technical Field

[0001] This invention belongs to the field of computational optical imaging technology, specifically relating to an adaptive filtering method for frequency division multiplexing ultrafast imaging reconstruction. Background Technology

[0002] Numerous ultrafast phenomena exist in nature, and observing these phenomena can help researchers better understand the fundamental mechanisms underlying these phenomena. Currently, researchers have proposed various ultrafast imaging methods, among which single-exposure multi-framing ultrafast imaging, with its advantage of capturing the entire dynamic scene in a single exposure, is particularly suitable for observing non-repeatable phenomena such as laser damage. Meanwhile, multi-exposure frequency recognition algorithms (FRAME), as a multi-framing frequency division multiplexing method, have wide applications due to their high temporal resolution and spectral adaptability.

[0003] However, due to the fundamental principle of frequency division multiplexing (FDM), researchers must repeatedly and manually select filter coordinates when applying this method. This manual selection introduces errors, leading to inconsistent reconstructed image quality. Furthermore, the influence of the zero-order term in the frame system and the multi-frame division necessitate positioning across multiple frames. The zero-order term also results in poor clustering and reduced accuracy, rendering traditional adaptive filtering methods used in off-axis digital holography unsuitable. Therefore, developing a novel adaptive filtering method for FDM ultrafast imaging reconstruction to improve the efficiency and spatial resolution of multi-exposure frequency recognition algorithms is of great significance. Summary of the Invention

[0004] To address the aforementioned problems in existing technologies, this invention proposes an adaptive filtering method for frequency division multiplexing ultrafast imaging reconstruction. This method is rationally designed, overcomes the shortcomings of existing technologies, and achieves good results.

[0005] To achieve the above objectives, the present invention adopts the following technical solution:

[0006] An adaptive filtering method for frequency division multiplexing ultrafast imaging reconstruction includes the following steps:

[0007] Step 1: Generate a random complex amplitude matrix and expand it into a random complex amplitude image using different interpolation methods. Generate random regions for every four images in chronological order. Apply sinusoidal stripe modulation in different directions using a Ronchi grating. After superposition, achieve multi-amplitude frequency division multiplexing to generate the original image.

[0008] Step 2: Based on the principle of multi-amplitude frequency division multiplexing ultrafast imaging system, remove the unmodulated random complex amplitude image from each original image in step 1 to obtain an image without zero-order terms, and combine it with the original image to construct a dataset;

[0009] Step 3: Improve the pix2pixGAN generative adversarial network and train the improved pix2pixGAN network;

[0010] Step 4: Input the actual multi-span frequency division multiplexing image captured using the frequency division multiplexing-based FRAME imaging method into the trained network, and output a multi-span frequency division multiplexing image without zero-level terms;

[0011] Step 5: Perform Fourier transform on the multi-amplitude frequency division multiplexing image without zero-order terms generated by the network to obtain the spectrum without zero-order terms, and then perform binarization on the spectrum.

[0012] Step 6: Apply the K-Means++ clustering algorithm to the binarized spectrum to obtain the center coordinates of the positive and negative first-order terms of the spectrum. Extract single-frame image information in the coordinate region using a Butterworth filter. Finally, perform inverse Fourier transform to obtain a frequency-division multiplexed single-frame image, thus achieving adaptive filtering.

[0013] Furthermore, step 1 includes the following sub-steps:

[0014] Step 1.1: Generate a complex amplitude matrix by generating a random matrix of size 2×2 to 10×10, randomly selecting values ​​0 to 15 as amplitude and 0 to 2π as phase;

[0015] Step 1.2: Extend the complex amplitude matrix to 256×256 using linear interpolation, nearest neighbor interpolation, and cubic spline interpolation respectively to obtain a large number of random complex amplitude matrix images;

[0016] Step 1.3: Add a random coefficient ranging from 0 to 0.5 to each complex amplitude matrix image to control the brightness and generate images with different brightness;

[0017] Step 1.4: Group the four images together and number them sequentially according to their generation order to simulate time changes. Select a random point as the initial point on the first image of each group. The next image will start from this initial point and generate an area with a random extension direction and random size. Randomly set the gray values ​​in this area to distinguish it from the background. Use the end point of this area as the initial point of the next image to continue generating random areas to simulate the different states of the dynamic scene at different times.

[0018] Step 1.5: Apply sinusoidal intensity stripe modulation in different directions to each of the four images in each group using a Ronchi grating to modulate each image to a specific position in the frequency domain. Then, superimpose the four modulated images in each group to obtain an image containing multiple stripe directions, thereby realizing multi-amplitude frequency division multiplexing and generating the initial simulation image of multi-amplitude frequency division multiplexing required for constructing the dataset.

[0019] Furthermore, step 1.5 specifically involves: a Ronchi grating. The expression is:

[0020] (1)

[0021] In the formula, and Let represent the intensity modulation coefficients of the Ronchi grating for modulation of the zero-order term and the intensity modulation coefficients for modulation of the carrier term, respectively. and These represent the Ronchi grating in... and Frequency modulation coefficient in the direction, and Represents image coordinates;

[0022] For object light containing dynamic scene information It can be expressed by the following formula:

[0023] (2)

[0024] In the formula, A represents the amplitude information of the dynamic scene, and φ represents the phase information of the dynamic scene;

[0025] Object light modulated by Langqi grating It can be expressed by the following formula:

[0026] (3)

[0027] As can be seen from the formula, the modulation of the Ronchi grating shifts the amplitude and phase information of the object light. and This manifests in the spectrum as the amplitude and phase information of the object light being modulated to a specific position in the spectrum. By applying sinusoidal stripe modulation in different directions to frame images at different times, images at different times can be modulated to different positions in the spectrum, and each frame image can be reconstructed through a bandpass filter.

[0028] Furthermore, in step 2, the unmodulated random complex amplitude image Represented as:

[0029] (4)

[0030] Multiply formula (4) by the modulation coefficient of the Ronchi grating. , and zero-level term expression The same, therefore After removing the zero-order terms from the multi-amplitude frequency division multiplexing ultrafast imaging image, an image without zero-order terms is obtained. At the same time, the brightness of the image without zero-order terms is adjusted to match that of the original image.

[0031] Furthermore, step 3 specifically involves: firstly, constructing the original pix2pixGAN network, which consists of a generator based on U-Net and a discriminator based on Patch-GAN. The U-Net generator includes two stages: upsampling and downsampling. Before the upsampling stage begins, a GAM attention mechanism module is added to assign weights to the features. After the upsampling stage ends and before the downsampling stage begins, another GAM attention mechanism module is introduced to enhance the network's ability to learn features while avoiding the problem of excessive training time cost caused by complex network structure and too many feature parameters.

[0032] The original images in the dataset are used as input to the improved pix2pixGAN network, and the corresponding images without zero-level terms are used as learning labels to train the improved pix2pixGAN network.

[0033] Furthermore, step 6 specifically includes the following sub-steps:

[0034] Step 6.1: Set the number of iterations n and the number of clusters. The number of iterations, n, is set empirically, balancing the clustering effect and algorithm complexity. The number of clusters... It is determined by the number of positive and negative first-level terms, i.e., carrier terms;

[0035] Step 6.2: Construct a dataset χ from the pixels with a value of 1 in the binarized spectrogram, and randomly select from the dataset χ. 100 sample points were used as the initial set of cluster centers. , For the first One initial cluster center;

[0036] Step 6.3: For the current set of cluster centers For each cluster center Perform the following operations to complete a cluster center update:

[0037] 6.3.1: Calculate the value of each sample point in the dataset χ. With cluster center shortest distance ;

[0038] 6.3.2: Calculate the probability that each sample point will be selected as the next cluster center. The formula is:

[0039] (5)

[0040] 6.3.3: Selection The largest sample point is used as the updated first... Cluster centers ;

[0041] 6.3.4: Complete all The update of each cluster center yields a new set of cluster centers. ;

[0042] Step 6.4: Update the set of cluster centers As the current set of cluster centers, repeat step 6.3 for a total of n iterations;

[0043] Step 6.5: After the iteration is complete, the final set of cluster centers is the one that satisfies the set number of iterations. There are 1 cluster center, and each cluster center corresponds to the center coordinates of a spectrum carrier term.

[0044] The beneficial technical effects of this invention are as follows:

[0045] This invention applies to the frequency division multiplexing-based FRAME method. By implementing zero-order term removal and adaptive filtering, it mitigates the impact of noise during reconstruction and eliminates errors caused by manually selecting filter coordinates, thereby improving the resolution of the reconstructed image. This enhances its performance in imaging ultrafast phenomena such as laser processing, facilitating better research into the fundamental mechanisms of ultrafast phenomena. Attached Figure Description

[0046] Figure 1 This is a flowchart of an adaptive filtering method for frequency division multiplexing ultrafast imaging reconstruction according to the present invention.

[0047] Figure 2 This is the multi-exposure frequency recognition algorithm system in this invention.

[0048] Figure 3 This is a schematic diagram illustrating the principle of zero-pole term removal in this invention;

[0049] Wherein, (a) is the original single-frame image; (b) is the frequency division multiplexed image; (c) is the image without grating modulation; (d) is the spectrum of the frequency division multiplexed image; (e) is the spectrum of the image without grating modulation; and (f) is the spectrum without zero-order terms.

[0050] Figure 4 The schematic diagram generated for the dataset of this invention;

[0051] Among them, (a)-(d) are four simulated images generated by different gratings and simulating inter-frame scene changes; (e) is the initial image of frequency division multiplexing obtained by superimposing the four images; and (f) is the image after removing the zero-order term from (e).

[0052] Figure 5 A comparison of image clustering results before and after removing level 0 terms;

[0053] Among them, (a)-(c) are the process and result diagrams for clustering images without removing zero-level terms; (d)-(f) are the process and result diagrams for clustering images after removing zero-level terms.

[0054] Figure 6 Example image for reconstructing a randomly generated image;

[0055] Among them, (a) is a randomly generated multiplexed image; (b) is the spectrum image generated by its Fourier transform; (c) is the spectrum image after binarization; (d) is the adaptive filtering result; (e)-(h) are the multiplexed image, spectrum image, binarization result and adaptive filtering result after removing the zero-level term of (a), respectively; and (i)-(l) are four single-frame images obtained by filtering and inverse Fourier transform after obtaining the filter center. Detailed Implementation

[0056] The specific embodiments of the present invention will be further described below with reference to specific examples:

[0057] An adaptive filtering method for frequency division multiplexing ultrafast imaging reconstruction, such as Figure 1 As shown, it includes the following steps:

[0058] Step 1: By generating a random complex amplitude matrix and expanding it into a random complex amplitude image using different interpolation methods, random regions are generated for every four images in chronological order. Sine stripe modulation in different directions is applied using a Ronchi grating, and the images are superimposed to achieve multi-amplitude frequency division multiplexing, generating a large number of original images for multi-amplitude frequency division multiplexing ultrafast imaging, in order to solve the problems of inconvenience in actual shooting, large workload, and poor generalization ability.

[0059] Step 1 includes the following sub-steps:

[0060] Step 1.1: Generate a complex amplitude matrix by generating a random matrix of size 2×2 to 10×10, randomly selecting values ​​0 to 15 as amplitude and 0 to 2π as phase;

[0061] Step 1.2: Extend the complex amplitude matrix to 256×256 using linear interpolation, nearest neighbor interpolation, and cubic spline interpolation respectively to obtain a large number of random complex amplitude matrix images;

[0062] Step 1.3: To enhance the generalization ability of the dataset, a random coefficient ranging from 0 to 0.5 is added to each complex amplitude matrix image to control the brightness, generating images with different brightness levels;

[0063] Step 1.4: To simulate the ultrafast imaging process, each set of four images is numbered sequentially according to their generation order to simulate time changes. A random point is selected as the initial point on the first image of each group. The next image uses this initial point as the starting point to generate a region with a random extension direction and random size. The gray values ​​within this region are then randomly reset to distinguish it from the background. The end point of this region is used as the starting point for the next image to continue generating random regions, thus simulating the different states of a dynamic scene at different times. This simulates the changes in the complex amplitude of the scene observed due to the time delay between frames during ultrafast imaging.

[0064] Step 1.5: Apply sinusoidal intensity stripe modulation in different directions to each of the four images in each group using a Ronchi grating to modulate each image to a specific position in the frequency domain. Then, superimpose the four modulated images in each group to obtain an image containing multiple stripe directions, thereby realizing multi-amplitude frequency division multiplexing and generating the initial simulation image of multi-amplitude frequency division multiplexing required for constructing the dataset.

[0065] Ronchi grating The expression is:

[0066] (1)

[0067] In the formula, and The intensity modulation coefficient represents the intensity modulation coefficient of the Ronchi grating for modulation of the zero-order term and the intensity modulation coefficient for modulation of the carrier term. and These represent the Ronchi grating in... and Frequency modulation coefficient in the direction, and Represents image coordinates;

[0068] For object light containing dynamic scene information It can be expressed by the following formula:

[0069] (2)

[0070] In the formula, A represents the amplitude information of the dynamic scene, and φ represents the phase information of the dynamic scene;

[0071] Object light modulated by Langqi grating It can be expressed by the following formula:

[0072] (3)

[0073] As can be seen from the formula, the modulation of the Ronchi grating shifts the amplitude and phase information of the object light. and This manifests in the spectrum as the amplitude and phase information of the object light being modulated to specific positions in the spectrum, achieving multi-amplitude frequency division multiplexing and solving the problem of difficult acquisition and low quality of actual images in dataset production. By applying sinusoidal fringe modulation in different directions to frame images at different times, images from different times can be modulated to different positions in the spectrum, and each frame image can be reconstructed through a bandpass filter.

[0074] However, due to the above formula The influence of the zero-order term limits the filter size, forcing a trade-off between using a larger filter radius to obtain more information and using a smaller filter radius to avoid introducing too much noise. In particular, when other adaptive filtering methods are applied to the system, the severe zero-order term influence can lead to serious deviations in carrier term positioning.

[0075] Step 2: Based on the principle of multi-amplitude frequency division multiplexing ultrafast imaging system, remove the unmodulated random complex amplitude image from each original image in step 1 to obtain an image without zero-order terms, and combine it with the original image to construct a dataset;

[0076] like Figure 3 The diagram illustrates the principle of zero-level term removal. For a given... Figure 3 The single-frame image shown in (a) is divided into frames to distinguish the transitions between frames. Figure 3 The image shown in (a) is rotated by 0°, 90°, 180°, and 270° respectively. Then, by applying Ronchi grating modulation in different directions to the images at different rotation angles and combining them, the following can be obtained: Figure 3 The frequency division multiplexing image shown in (b) is illustrated. Its corresponding spectral image is shown below. Figure 3 As shown in (d).

[0077] Figure 3 In the middle (c), the intensity image is unmodulated, defined as... Represented as:

[0078] (4)

[0079] Multiply formula (4) by the modulation coefficient of the Ronchi grating. , and zero-level term expression Same, from Figure 3 The spectrum of the unmodulated image shown in (e) can also be seen to be its spectrum. Figure 3 The zero-level term in (d). Therefore, it will be as follows. Figure 3 The unmodulated intensity image shown in (d) is derived from a multi-amplitude frequency division multiplexing ultrafast imaging image ( Figure 3 After removing the term from (b), the zero-level term at the center is removed, resulting in an image without zero-level terms. The spectrum of the image without zero-level terms is as follows: Figure 3As shown in (f), the spectrum does not contain zero-order term noise, thus eliminating the influence of zero-order terms on the reconstruction.

[0080] Based on the above principle, zero-order terms can be removed from the image generated in step 1, such as... Figure 4 As shown, generate the following based on the content described in step 1: Figure 4 The set of varied images shown in (a)-(d) contains four images modulated by gratings in different directions, and it can be seen that... Figure 4 (b) in Figure 4 Based on (a), modify the background of some areas, and so on. Figure 4 In the middle (d), the image changes caused by dynamic events are simulated. These are then superimposed to obtain the image as shown. Figure 4 The initial image for frequency division multiplexing is shown in (e). The zero-order term removal principle described above is applied to... Figure 4 Processing (e) yields the following result: Figure 4 The image without zero-order terms is shown in (f). This step is repeated to generate a large number of randomly different paired training images, including multi-amplitude frequency division multiplexing ultrafast imaging images and images without zero-order terms, to replace actual paired images for training the network, thus solving the problems of difficulty in acquiring actual images and poor zero-order term removal.

[0081] Step 3: Improve the pix2pixGAN generative adversarial network to remove zero-level terms. Introduce the GAM attention mechanism in the upsampling and downsampling parts of the generator U-Net network to enhance the network's ability in the zero-level term removal task. Use the simulated multi-amplitude frequency division multiplexing ultrafast imaging original image as the input of the improved pix2pixGAN network, and the corresponding image without zero-level terms as the learning label to train the improved pix2pixGAN network.

[0082] Specifically, the pix2pixGAN network is a generative adversarial network based on deep learning methods. Its basic principle can be expressed by the following equation:

[0083] (6)

[0084] The Pix2pixGAN network is a neural network that learns to map from an input image to an output image. It consists of a generator G and a discriminator D. The generator G uses a U-Net architecture and comprises convolutional layers, normalization layers, activation layers, and deconvolutional layers. The generator updates its network parameters using adversarial loss and a weighted L1 loss between the generated image and the expected output image. The discriminator D uses a Patch-GAN model, consisting of convolutional layers and activation layers. It divides the input image into blocks of a specific size and then distinguishes between real and fake images. The network's ultimate goal is a game between the generator and the discriminator. The generator attempts to generate images that closely resemble real images to "deceive" the discriminator, while the discriminator strives to improve its performance to better distinguish between real and generated images, thus achieving high-quality image-to-image translation.

[0085] The GAM attention mechanism achieves a similar focusing effect to the human eye by weighting specific channel and location-based feature information. The GAM attention mechanism includes a channel attention submodule and a spatial attention submodule. The channel attention submodule preserves information in three dimensions using a three-dimensional arrangement, and then amplifies cross-dimensional channel spatial dependencies using a two-layer multilayer perceptron. The spatial attention submodule uses two convolutional layers to fuse spatial information. It performs max pooling and average pooling on the feature maps, stacks them together, performs convolution, and then applies an activation function to achieve the spatial attention mechanism.

[0086] The GAM attention mechanism can be represented by the following formula:

[0087] (7)

[0088] (8)

[0089] In the formula, For a given input feature mapping, Indicates an intermediate state. Indicates the output result. and These are channel attention maps and spatial attention maps, respectively. This indicates that element-wise multiplication is performed.

[0090] First, the original pix2pixGAN network is built. The original pix2pixGAN network consists of a generator based on U-Net and a discriminator based on Patch-GAN. The U-Net generator includes two stages: upsampling and downsampling. A GAM attention mechanism module is added before the upsampling stage to assign weights to the features before upsampling. After the upsampling stage ends and before the downsampling stage begins, another GAM attention mechanism module is introduced to enhance the network's ability to learn features while avoiding the problem of excessive training time cost caused by complex network structure and too many feature parameters.

[0091] like Figure 4 The diagram illustrates the impact of removing zero-level terms on clustering performance. Specifically, crosstalk from zero-level terms leads to high-intensity interference terms at the center of the spectrum. Without removing zero-level terms, these interference terms result in an increase in data points and errors in cluster distance calculation, making it impossible to correctly select the coordinates of the positive and negative first-level terms. However, removing zero-level terms eliminates the interference terms at the center of the spectrum, improving the accuracy and effectiveness of clustering.

[0092] Step 4: Input the actual multi-amplitude frequency division multiplexing (MPFDM) images captured using the FRAME method of the MPFDM system into the trained network. The network outputs MPFDM images without zero-order terms.

[0093] like Figure 2 As shown, the principle of the FRAME method in a multi-amplitude frequency division multiplexing system is as follows:

[0094] The input light is split into four beams, each of which is modulated by a Ronchi grating in a different direction. The delay module is adjusted to give each beam a different time code. After encoding, the four beams are combined to illuminate dynamic events, resulting in a multi-amplitude frequency division multiplexing ultrafast imaging image. The dynamic scene at different times illuminated by each sub-beam is modulated to a specific frequency domain. The spatial frequency locking algorithm can be used to separate each frame of the image.

[0095] Step 5: Perform Fourier transform on the multi-amplitude frequency division multiplexing image without zero-order terms generated by the network to obtain the spectrum without zero-order terms, and then perform binarization on the spectrum.

[0096] Step 6: Apply the K-Means++ clustering algorithm to the binarized spectrum to obtain the center coordinates of the positive and negative first-order terms of the spectrum. Extract the single-frame image information corresponding to the center coordinates in the coordinate region through the Butterworth filter. Finally, perform inverse Fourier transform to obtain the frequency division multiplexed single-frame image and realize adaptive filtering.

[0097] Compared to the traditional K-Means algorithm, the K-Means++ algorithm does not require manual selection of initial cluster coordinates, thus improving the algorithm's efficiency.

[0098] Step 6 specifically includes the following sub-steps:

[0099] Step 6.1: Set the number of iterations n and the number of clusters. The number of iterations, n, is set empirically, balancing the clustering effect and algorithm complexity. The number of clusters... It is determined by the number of positive and negative first-level terms, i.e., carrier terms;

[0100] Step 6.2: Construct a dataset χ from the pixels with a value of 1 in the binarized spectrogram, and randomly select from the dataset χ. 100 sample points were used as the initial set of cluster centers. , For the first One initial cluster center;

[0101] Step 6.3: For the current set of cluster centers For each cluster center Perform the following operations to complete a cluster center update:

[0102] 6.3.1: Calculate the value of each sample point in the dataset χ. With cluster center shortest distance ;

[0103] 6.3.2: Calculate the probability that each sample point will be selected as the next cluster center. The formula is:

[0104] (5)

[0105] 6.3.3: Selection The largest sample point is used as the updated first... Cluster centers ;

[0106] 6.3.4: Complete all The update of each cluster center yields a new set of cluster centers. ;

[0107] Step 6.4: Update the set of cluster centers As the current set of cluster centers, repeat step 6.3 for a total of n iterations;

[0108] Step 6.5: After the iteration is complete, the final set of cluster centers is the one that satisfies the set number of iterations. There are 1 cluster center, and each cluster center corresponds to the center coordinates of a spectrum carrier term.

[0109] like Figure 5 Figures (a) to (f) illustrate the impact of removing zero-level terms on clustering performance. Specifically, crosstalk from zero-level terms leads to high-intensity interference terms at the center of the spectrum. Without removing zero-level terms, these interference terms result in an increase in data points and errors in cluster distance calculation, making it impossible to correctly select the coordinates of the positive and negative first-level terms. Removing zero-level terms eliminates the interference terms at the center of the spectrum, improving the accuracy and effectiveness of clustering.

[0110] Figure 6Figures (a) to (l) illustrate an application example of this invention. Adaptive filtering is applied to the simulated multi-amplitude frequency division multiplexing ultrafast imaging image. The generated original image is input into the improved pix2pixGAN network described in this invention, generating an image without zero-order terms as shown in the figure. Fourier transforms of both images also show significant suppression of zero-order terms in the spectrum. After removing the zero-order terms, binarization reveals that the interference terms at the center of the spectrum are completely removed. Clustering using the K-Means++ algorithm accurately locates the coordinate centers of the positive and negative first-order terms, achieving adaptive filtering. After locating the center coordinates, bandpass filtering is applied to these coordinates to separate and reconstruct a single-frame image.

[0111] Of course, the above description is not intended to limit the present invention, and the present invention is not limited to the examples given above. Any changes, modifications, additions or substitutions made by those skilled in the art within the scope of the present invention should also fall within the protection scope of the present invention.

Claims

1. An adaptive filtering method for frequency division multiplexing ultrafast imaging reconstruction, characterized in that, Includes the following steps: Step 1: Generate a random complex amplitude matrix and expand it into a random complex amplitude image using different interpolation methods. Generate random regions for every four images in chronological order. Apply sinusoidal stripe modulation in different directions using a Ronchi grating. After superposition, achieve multi-amplitude frequency division multiplexing to generate the original image. Step 2: Based on the principle of multi-amplitude frequency division multiplexing ultrafast imaging system, remove the unmodulated random complex amplitude image from each original image in step 1 to obtain an image without zero-order terms, and combine it with the original image to construct a dataset; Step 3: Improve the pix2pixGAN generative adversarial network and train the improved pix2pixGAN network; Step 3 specifically involves: First, building the original pix2pixGAN network, which consists of a generator based on U-Net and a discriminator based on Patch-GAN. The U-Net generator includes two stages: upsampling and downsampling. Before the upsampling stage begins, a GAM attention mechanism module is added to assign weights to the features. After the upsampling stage ends and before the downsampling stage begins, another GAM attention mechanism module is introduced to enhance the network's ability to learn features while avoiding the problem of excessive training time cost caused by complex network structure and too many feature parameters. The original images in the dataset are used as input to the improved pix2pixGAN network, and the corresponding images without zero-level terms are used as learning labels to train the improved pix2pixGAN network. Step 4: Input the actual multi-span frequency division multiplexing image captured using the frequency division multiplexing-based FRAME imaging method into the trained network, and output a multi-span frequency division multiplexing image without zero-level terms; Step 5: Perform Fourier transform on the multi-amplitude frequency division multiplexing image without zero-order terms generated by the network to obtain the spectrum without zero-order terms, and then perform binarization on the spectrum. Step 6: Apply the K-Means++ clustering algorithm to the binarized spectrum to obtain the center coordinates of the positive and negative first-order terms of the spectrum. Extract single-frame image information in the coordinate region using a Butterworth filter. Finally, perform inverse Fourier transform to obtain a frequency-division multiplexed single-frame image, thus achieving adaptive filtering.

2. The adaptive filtering method for frequency division multiplexing ultrafast imaging reconstruction according to claim 1, characterized in that, Step 1 includes the following sub-steps: Step 1.1: Generate a complex amplitude matrix by generating a random matrix of size 2×2 to 10×10, randomly selecting values ​​0 to 15 as amplitude and 0 to 2π as phase; Step 1.2: Extend the complex amplitude matrix to 256×256 using linear interpolation, nearest neighbor interpolation, and cubic spline interpolation respectively to obtain a large number of random complex amplitude matrix images; Step 1.3: Add a random coefficient ranging from 0 to 0.5 to each complex amplitude matrix image to control the brightness and generate images with different brightness; Step 1.4: Group the four images together and number them sequentially according to their generation order to simulate time changes. Select a random point as the initial point on the first image of each group. The next image will start from this initial point and generate an area with a random extension direction and random size. Randomly set the gray values ​​in this area to distinguish it from the background. Use the end point of this area as the initial point of the next image to continue generating random areas to simulate the different states of the dynamic scene at different times. Step 1.5: Apply sinusoidal intensity stripe modulation in different directions to each of the four images in each group using a Ronchi grating to modulate each image to a specific position in the frequency domain. Then, superimpose the four modulated images in each group to obtain an image containing multiple stripe directions, thereby realizing multi-amplitude frequency division multiplexing and generating the initial simulation image of multi-amplitude frequency division multiplexing required for constructing the dataset.

3. The adaptive filtering method for frequency division multiplexing ultrafast imaging reconstruction according to claim 2, characterized in that, Step 1.5 specifically involves: Ronchi grating The expression is: ； (1) In the formula, and Let represent the intensity modulation coefficients of the Ronchi grating for modulation of the zero-order term and the intensity modulation coefficients for modulation of the carrier term, respectively. and These represent the Ronchi grating in... and Frequency modulation coefficient in the direction, and Represents image coordinates; For object light containing dynamic scene information It can be expressed by the following formula: ；(2) In the formula, A represents the amplitude information of the dynamic scene, and φ represents the phase information of the dynamic scene; Object light modulated by Langqi grating It can be expressed by the following formula: ；(3) As can be seen from the formula, the modulation of the Ronchi grating shifts the amplitude and phase information of the object light. and This manifests in the spectrum as the amplitude and phase information of the object light being modulated to a specific position in the spectrum. By applying sinusoidal stripe modulation in different directions to frame images at different times, images at different times can be modulated to different positions in the spectrum, and each frame image can be reconstructed through a bandpass filter.

4. The adaptive filtering method for frequency division multiplexing ultrafast imaging reconstruction according to claim 2, characterized in that, In step 2, the unmodulated random complex amplitude image Represented as: ； (4) Multiply formula (4) by the modulation coefficient of the Ronchi grating. , and zero-level term expression The same, therefore After removing the zero-order terms from the multi-amplitude frequency division multiplexing ultrafast imaging image, an image without zero-order terms is obtained. At the same time, the brightness of the image without zero-order terms is adjusted to match that of the original image.

5. The adaptive filtering method for frequency division multiplexing ultrafast imaging reconstruction according to claim 1, characterized in that, Step 6 specifically includes the following sub-steps: Step 6.1: Set the number of iterations n and the number of clusters. The number of iterations, n, is set empirically, balancing the clustering effect and algorithm complexity. The number of clusters... It is determined by the number of positive and negative first-level terms, i.e., carrier terms; Step 6.2: Construct a dataset χ from the pixels with a value of 1 in the binarized spectrogram, and randomly select from the dataset χ. 100 sample points were used as the initial set of cluster centers. , For the first One initial cluster center; Step 6.3: For the current set of cluster centers For each cluster center Perform the following operations to complete a cluster center update: 6.3.1: Calculate the value of each sample point in the dataset χ. With cluster center shortest distance ; 6.3.2: Calculate the probability that each sample point will be selected as the next cluster center. The formula is: ；(5) 6.3.3: Selection The largest sample point is used as the updated first... Cluster centers ; 6.3.4: Complete all The update of each cluster center yields a new set of cluster centers. ; Step 6.4: Update the set of cluster centers As the current set of cluster centers, repeat step 6.3 for a total of n iterations; Step 6.5: After the iteration is complete, the final set of cluster centers is the one that satisfies the set number of iterations. There are 1 cluster center, and each cluster center corresponds to the center coordinates of a spectrum carrier term.

Images