A method for improving the detection sensitivity of a streak camera and related apparatus

Abstract

The application relates to the technical field of image processing, and particularly provides a method for improving the detection sensitivity of a stripe camera and related equipment, which comprises the following steps: acquiring multiple frames of background images and signal-containing images of the stripe camera under the same imaging condition, and determining a standardization sequence for each pixel; performing statistical feature extraction on the standardization sequence for each pixel to construct a pixel-level statistical feature tensor; splicing the pixel-level statistical feature tensor and noise condition features to form a pixel-level joint feature vector; constructing a U-Net network with residual learning introduced; and performing enhancement reconstruction on the pixel-level joint feature vector based on the completed U-Net network to obtain an enhancement result. After the background noise is modeled and normalized pixel by pixel, the pixel-level statistical feature tensor and the pixel noise standard deviation are taken as network condition inputs, so that the enhancement intensity is adaptively adjusted according to the noise level, the structural details are maintained, and strong noise suppression and weak signal enhancement are realized.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a method and related equipment for improving the detection sensitivity of a stripe camera. Background Technology

[0002] Striking cameras are the only scientific measurement and diagnostic instruments that simultaneously possess ultra-high resolution (fs-ps level) and high spatial resolution (µm level), making them essential for detecting microscopic and ultrafast processes. Currently, synchronous scanning striking cameras, with their high repetition rate and high gain, have achieved ultrafast low-light measurements and are widely used in laser diagnostics, transient phenomenon observation, and fluorescence lifetime measurements. However, under extremely weak light conditions approaching the single-photon level, the number of effective photons in a single frame is extremely small, and the target signal exhibits a sparse and random distribution in the spatial domain. Simultaneously, under extremely weak light conditions, very high microchannel plate gain is required, which significantly enhances system dark noise, shot noise, and readout noise, resulting in an extremely low signal-to-noise ratio for a single frame and making it difficult to extract weak signals.

[0003] Existing methods for improving the detection sensitivity of streak cameras under extremely low light conditions mostly employ cumulative enhancement strategies such as multi-frame analog integration or centroid methods. However, under extremely low light conditions, the target signal intensity is too weak and easily overwhelmed by noise. Analog integration accumulates noise along with the signal, resulting in the weak signal not being effectively extracted in the final result. The centroid method only considers the camera's readout noise and does not take into account dark noise and shot noise that increase with gain amplification, resulting in limited improvement in image signal-to-noise ratio and making it difficult to meet the requirements of single-photon level measurement and high-sensitivity detection. Summary of the Invention

[0004] To address the aforementioned issues, this application provides a method and related equipment for improving the detection sensitivity of a streak camera. The method uses the pixel-level statistical feature tensor extracted through statistical features and the pixel noise standard deviation guided by noise conditions as the conditional input of a multi-scale fully convolutional network that incorporates residual learning. This allows the enhancement intensity to be adaptively adjusted according to the noise level, achieving strong noise suppression and weak signal enhancement while preserving structural details.

[0005] To achieve the objectives of this application, the following technical solution is provided: In a first aspect, this application provides a method for improving the detection sensitivity of a streak camera, comprising: Multiple frames of background images and signal-containing images are acquired by a stripe camera under the same imaging conditions. A normalized sequence is determined for each pixel based on the background images and the signal-containing images. The normalized sequence is the pixel sequence value after eliminating the influence of a fixed background for each pixel. For each pixel, statistical features are extracted from the standardized sequence to construct a pixel-level statistical feature tensor; at the same time, the standard deviation of pixel noise is logarithmically compressed and used as a noise conditional feature, and the pixel-level statistical feature tensor and the noise conditional feature are concatenated in the channel dimension to form a pixel-level joint feature vector. A U-Net network incorporating residual learning is constructed. The decoder of the U-Net network is a multi-scale hierarchical structure symmetrical to the encoder. The encoder and the decoder are fused with features of the same scale through skip connections. The pixel-level joint feature vector is enhanced and reconstructed based on the trained U-Net network to obtain the enhanced result.

[0006] A further improvement of this application is that the acquisition of multiple frames of background images and signal-containing images from the stripe camera under the same imaging conditions, and the determination of a normalized sequence for each pixel based on the background images and the signal-containing images, includes: acquiring multiple frames of background images and signal-containing images from the stripe camera under conditions of no incident signal. The pixel intensity sequence of the frame background image, and... Perform pixel-by-pixel statistics on the background image of the frame, and calculate the position of each pixel in the frame. Mean background noise and standard deviation of pixel noise in the frame background image; acquisition The pixel intensity sequence of the frame containing the signal image is used to determine the signal intensity value for each pixel coordinate. For each pixel coordinate, pixel-level background noise removal and noise normalization are performed based on the mean of the background noise, the standard deviation of the pixel noise, and the signal intensity value to obtain a normalized sequence.

[0007] A further improvement of this application is that the acquisition... The frame contains a sequence of pixel intensity values ​​of a signal image. Determining the signal intensity value for each pixel coordinate includes: acquiring... Pixel intensity sequence of a frame containing a signal image , For the first The original signal-containing image in coordinates The total intensity value of the pixel; wherein, the photon signal is described by a heteroscedasticity model based on photon counting and readout Gaussian noise, and the heteroscedasticity model is specifically: ; In the formula, For the first Frame containing signal image in coordinates The number of photons reaching the pixel at that location is [number]. for The image frame number containing the signal image. For the first Frame containing signal image in coordinates The photon arrival count at a location follows the parameter: The Poisson distribution, For the first Frame containing signal image in coordinates The average photon arrival rate at the location; Gain / response coefficient; coordinates A fixed background item at a given pixel; For the first Frame containing signal image in coordinates Readout noise value at the location, , refers to the Frame containing signal image in coordinates The readout noise term at point follows a mean of 0 and a variance of . The normal distribution coordinates The readout noise variance corresponding to the pixel at that location.

[0008] A further improvement of this application is that, for each pixel coordinate, pixel-level background noise removal and noise normalization are performed based on the mean background noise, the standard deviation of pixel noise, and the signal strength value to obtain a normalized sequence, including: for each pixel coordinate: ; In the formula, For the first Frame containing signal image in coordinates The pixel sequence values ​​after standardization. For the first The original signal-containing image of the frame in coordinates The total intensity value of the pixel at that location. coordinates Mean background noise of the pixel coordinates The standard deviation of pixel noise at a given pixel; To prevent stable terms with a denominator of zero, .

[0009] A further improvement in this application is that the step of extracting statistical features from the standardized sequence and constructing a pixel-level statistical feature tensor includes: for each pixel position, extracting the statistical features of a single pixel position from the standardized sequence and constructing a pixel-level statistical feature tensor. All standardized sequences in a frame-containing signal image are treated as a set of time series, and time mean features, time variance features, global time series maximum features, and effective frame count features are constructed. The time mean features, time variance features, global time series maximum features, and effective frame count features are concatenated by channel to obtain a pixel-level statistical feature tensor. The step of using the pixel noise standard deviation of the pixel as a noise conditional feature after logarithmic compression, and concatenating the pixel-level statistical feature tensor and the noise conditional feature in the channel dimension to form a pixel-level joint feature vector, includes: using the pixel noise standard deviation of each pixel as a noise conditional feature. As a conditional input, and subjected to dynamic range compression mapping, specifically: ; in, The noise condition characteristics after compression. For logarithmic mapping, The standard deviation of noise. It is a stable term; ; in, It is a pixel-level joint feature vector. For the pixel-level statistical feature tensor, This is a splicing process at the channel dimension.

[0010] A further improvement of this application is that the U-Net network includes an encoder, a bottleneck layer, and a decoder; the encoder is a multi-scale hierarchical structure, selecting four scale levels, with the number of feature channels increasing progressively at each level, each level including at least two convolutional operations, followed by a non-linear activation function, and zero-padding is used to maintain the feature map size at the convolution kernel size; the bottleneck layer is located at the end of the encoder and consists of multiple convolutional structures; the decoder is a multi-scale hierarchical structure symmetrical to the encoder, restoring the spatial resolution of the feature map through progressive upsampling; wherein, the upsampling operation is implemented by transposed convolution or a combination of interpolation upsampling and convolution; the U-Net network learns the differential residual information between the baseline input image and the target enhancement result, and the network output is represented as:

[0011] in, The input tensor is composed of the pixel-level joint feature vectors. For the mapping of the neural network to be trained, For the network at pixel location The residual components output at the location.

[0012] Enhanced results Represented as:

[0013] in, As a reference, the input image is located at the pixel position. Pixel value at that location, To enhance the results.

[0014] A further improvement of this application is that, during the training phase, the U-Net network divides the pixel-level joint feature vector into at least two sub-sequence groups, forming a first pixel-level joint feature sub-vector and a second pixel-level joint feature sub-vector. The first pixel-level joint feature sub-vector and the second pixel-level joint feature sub-vector are then input into the U-Net network, and training constraints are applied using a constructed consistency loss function. Specifically, the consistency loss function is: ; or: ; In the formula, For consistency loss function, This is the first output result corresponding to the first pixel-level joint feature vector. This is the second output result corresponding to the second pixel-level joint feature vector. for Norm, for Norm; During training, gradient descent-based optimization methods are used to update network parameters.

[0015] A further improvement of this application is that it also includes: when using the consistency loss function for training constraints, introducing smoothness constraints and sparsity constraints, specifically: ; in, For the total loss function, For consistency loss function, For the total variational smoothing term, For sparse constraint terms, and These are the weighting coefficients.

[0016] Secondly, this application provides an apparatus for improving the detection sensitivity of a stripe camera, used to implement the above-described method for improving the detection sensitivity of a stripe camera, the apparatus comprising: The sequence acquisition module is used to acquire multiple frames of background images and signal-containing images from the streak camera under the same imaging conditions, and to determine a normalized sequence for each pixel based on the background images and signal-containing images; the normalized sequence is the pixel sequence value after eliminating the influence of a fixed background for each pixel. The feature extraction module is used to extract statistical features from the standardized sequence for each pixel and construct a pixel-level statistical feature tensor. At the same time, the standard deviation of pixel noise is logarithmically compressed and used as a noise condition feature. The pixel-level statistical feature tensor and the noise condition feature are concatenated in the channel dimension to form a pixel-level joint feature vector. The residual learning module constructs a U-Net network that incorporates residual learning. The decoder of the U-Net network is a multi-scale hierarchical structure symmetrical to the encoder. The encoder and the decoder fuse features at the same scale through skip connections. The result generation module is used to enhance and reconstruct the pixel-level joint feature vector based on the trained U-Net network to obtain the enhanced result.

[0017] Thirdly, this application provides a terminal, the terminal including a memory and one or more processors; the memory stores one or more programs; the programs include instructions for executing a method for improving the detection sensitivity of a streak camera as described above; the processor is used to execute the programs.

[0018] Compared with the prior art, the present invention has the following beneficial effects: This application provides a method and related equipment for improving the detection sensitivity of a streak camera. The method involves acquiring multiple frames of background and signal-containing images from the streak camera under the same imaging conditions. Based on the background and signal-containing images, a standardized sequence is determined for each pixel. Statistical features are extracted from the standardized sequence to construct a pixel-level statistical feature tensor. Simultaneously, the pixel noise standard deviation of the pixel is logarithmically compressed and used as a noise conditional feature. This noise conditional feature is then concatenated with the pixel-level statistical feature tensor along the channel dimension to form a pixel-level joint feature vector. Finally, the pixel-level joint feature vector is enhanced and reconstructed using a U-Net network incorporating residual learning to obtain the enhanced result. Therefore, in extremely low light and low single-frame signal-to-noise ratio scenarios, the influence of fixed background noise is effectively weakened and the noise scale is unified by first performing pixel-by-pixel statistical modeling and normalization of background noise, thereby improving the separability of weak photon events. Then, the pixel-level statistical feature tensor extracted through statistical features and the pixel noise standard deviation guided by noise conditions are used as the conditional input of a multi-scale fully convolutional network that introduces residual learning, so that the enhancement intensity can be adaptively adjusted with the noise level to adapt to different gain, illumination conditions and experimental parameter settings, thereby achieving strong noise suppression and weak signal enhancement while maintaining structural details. Attached Figure Description

[0019] The accompanying drawings are provided to further understand this application and form part of the specification. They are used together with the embodiments of this application to explain this application and do not constitute a limitation thereof. Figure 1 This is a schematic diagram of the working link and noise distribution of a stripe camera provided in an embodiment of this application; Figure 2 A prior art simulation integration principle diagram provided for embodiments of this application; Figure 3A schematic diagram of the prior art centroid method provided for embodiments of this application; Figure 4 A schematic diagram of an optional first process for improving the detection sensitivity of a streak camera, provided as an embodiment of this application; Figure 5 A schematic diagram of an optional second process for improving the detection sensitivity of a streak camera, provided as an embodiment of this application; Figure 6 This is a schematic diagram of the U-Net network structure provided in an embodiment of this application; Figure 7 A schematic diagram of fluorescence signal intensity at the centroid method under 9ns level and LD15 conditions provided in the embodiments of this application; Figure 8 A schematic diagram of the fluorescence signal intensity of the method of this application under the 9ns level and LD15 conditions provided for embodiments of this application; Figure 9 A schematic diagram of the fluorescence decay curve of the centroid method under 9ns setting and LD15 conditions provided in the embodiments of this application; Figure 10 A schematic diagram of the fluorescence decay curve of the method of this application under 9ns level and LD15 conditions provided for the embodiments of this application; Figure 11 A schematic diagram of fluorescence signal intensity under centroid method at 9ns level and LD11 conditions provided in the embodiments of this application; Figure 12 A schematic diagram of the fluorescence signal intensity of the method of this application under the 9ns level and LD11 conditions provided for embodiments of this application; Figure 13 A schematic diagram of the fluorescence decay curve of the centroid method under LD11 conditions at the 9ns level, provided in an embodiment of this application; Figure 14 A schematic diagram of the fluorescence decay curve of the method of this application under 9ns level and LD11 conditions provided for the embodiments of this application; Figure 15 A schematic diagram of fluorescence signal intensity at the centroid method under LD8 conditions at the 9ns level, provided in an embodiment of this application; Figure 16 A schematic diagram of the fluorescence signal intensity of the method of this application under 9ns level and LD8 conditions, provided for embodiments of this application; Figure 17 A schematic diagram of the fluorescence decay curve of the centroid method under LD8 conditions at the 9ns level, provided in the embodiments of this application; Figure 18 A schematic diagram of the fluorescence decay curve of the method of this application under 9ns level and LD8 conditions, provided for the embodiments of this application; Figure 19A schematic diagram of fluorescence signal intensity at the centroid method under LD15 conditions at the 51ns level, provided for an embodiment of this application; Figure 20 A schematic diagram of the fluorescence signal intensity of the method of this application under the conditions of 51ns level and LD15, provided for the embodiments of this application; Figure 21 A schematic diagram of the fluorescence decay curve of the centroid method under LD15 conditions at the 51ns level, provided for an embodiment of this application; Figure 22 A schematic diagram of the fluorescence decay curve of the method of this application under 51ns level and LD15 conditions provided for the embodiments of this application; Figure 23 A schematic diagram of fluorescence signal intensity at the centroid method under LD11 conditions at the 51ns level, provided in an embodiment of this application; Figure 24 A schematic diagram of the fluorescence signal intensity of the method of this application under 51ns level and LD11 conditions provided for embodiments of this application; Figure 25 A schematic diagram of the fluorescence decay curve of the centroid method under LD11 conditions at the 51ns level, provided for an embodiment of this application; Figure 26 A schematic diagram of the fluorescence decay curve of the method of this application under LD11 conditions at a 51ns level, provided for an embodiment of this application; Figure 27 A schematic diagram of fluorescence signal intensity at the 51ns level and under LD8 conditions provided in this application embodiment; Figure 28 A schematic diagram of the fluorescence signal intensity of the method of this application under LD8 conditions at a 51ns level, provided for an embodiment of this application; Figure 29 A schematic diagram of the fluorescence decay curve of the centroid method under LD8 conditions at the 51ns level, provided in an embodiment of this application; Figure 30 A schematic diagram of the fluorescence decay curve of the method of this application under LD8 conditions at a 51ns level, provided for an embodiment of this application; Figure 31 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0021] The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature; in the description of this application, unless otherwise stated, "multiple" means two or more.

[0022] A streak camera (Streak-Camera Low-Light Signal Extractor, SC-LLSE) is the only scientific measurement and diagnostic instrument that simultaneously possesses ultra-high resolution (fs-ps level) and high spatial resolution (um level), making it an essential means of probing microscopic and ultrafast processes. The T-lab series of general-purpose streak cameras, as precision instruments for measuring the temporal characteristics of ultrafast light pulses, achieves picosecond to femtosecond temporal resolution through a "time-space" conversion. Figure 1 As shown, its specific working process is as follows: The light signal to be measured (wavelength range 200–900 nm) enters the system through a slit and first illuminates the photocathode of the streak image inverter tube, generating photoelectrons. These photoelectrons are accelerated and focused by a high-voltage electric field to form an electron beam, which enters the scanning deflection region. At this time, a high-speed linear voltage (scanning frequency up to 200 MHz or more) applied to the deflection electrodes causes the electron beam to deflect linearly with time in the vertical direction—thus converting time information into spatial position. Subsequently, the electron beam bombards the fluorescent screen to excite fluorescence, forming a two-dimensional image, where the horizontal direction corresponds to spectral or spatial information, and the vertical direction corresponds to the time distribution. Finally, the image is acquired by a scientific-grade CCD or CMOS camera, and the high-resolution waveform or time-resolved spectrum of light intensity changing with time is reconstructed by software. This camera integrates both single-trigger and high-frequency scanning modes, combining high temporal resolution with ease of operation, and is suitable for various scientific research and industrial applications such as ultrafast laser diagnostics, fluorescence lifetime imaging, and time-resolved spectroscopy.

[0023] Currently, synchronous scanning streak cameras, with their high repetition rate and high gain, enable ultrafast low-light measurements and are widely used in fields such as laser diagnostics, transient phenomenon observation, and fluorescence lifetime. However, under extremely weak light conditions approaching the single-photon level, the number of effective photons in a single frame image is extremely small, and the target signal exhibits a sparse and random distribution in the spatial domain. Simultaneously, under extremely weak light conditions, a very high microchannel plate (MCP) gain is required, which significantly enhances system dark noise, shot noise, and readout noise, resulting in an extremely low signal-to-noise ratio (SNR) for a single frame image, making it difficult to extract weak signals (e.g., ...). Figure 1 ).

[0024] Existing methods for improving the detection sensitivity of streak cameras under extremely low light conditions mostly employ cumulative enhancement strategies such as multi-frame simulated integration or centroid methods. However, under extremely low light conditions, reference... Figure 2 As shown, because the target signal is too weak and easily overwhelmed by noise, analog integration accumulates noise along with the signal, resulting in the weak signal not being effectively extracted in the final result; while the centroid rule only considers the camera's readout noise, referring to... Figure 3 As shown, it does not take into account the dark noise and shot noise that increase with gain amplification, resulting in limited SNR improvement and making it difficult to meet the requirements of single-photon level measurement and high-sensitivity detection.

[0025] To address the aforementioned technical problems, the present invention proposes the following technical solutions and corresponding embodiments based on the above-mentioned stripe camera.

[0026] The following is combined Figures 4 to 31 The illustrated embodiments describe the technical solution of the present invention: Example 1 This application provides a method for improving the detection sensitivity of a stripe camera, referring to... Figure 4 As shown, the process includes the following steps S101 to S104: Step S101: Acquire multiple frames of background images and signal-containing images from the stripe camera under the same imaging conditions, and determine a normalized sequence for each pixel based on the background images and the signal-containing images; the normalized sequence is the pixel sequence value after eliminating the influence of a fixed background for each pixel.

[0027] In this embodiment, multiple frames of background images are acquired under conditions of no incident signal, and pixel-by-pixel statistics are performed on each pixel location to calculate the mean background noise and the standard deviation of pixel noise. Specifically, refer to... Figure 5 Under conditions of no incident signal, data is collected. Frame background image Its pixel intensity can be expressed as: ;in, for The image frame number of the background image. For the first Frame background image in coordinates The actual intensity value of the pixel at that location. coordinates The fixed background base (fixed background / bias term) of the pixel. For the first Frame background image in coordinates The random noise component (random noise term) at each pixel location is then analyzed pixel-by-pixel, calculating the random noise component at each pixel location. Mean background noise of the frame background image and pixel noise standard deviation : ; ; Among them, the standard deviation of pixel noise It is used to quantify the inherent noise intensity of the system under the current operating parameters, can be used to fix the background noise term for correction / suppression, and provide a basis for the noise level for the generalization of the subsequent model under different gain, light intensity and experimental parameter settings.

[0028] Simultaneously, collection Frame containing signal sequence , For the first The original signal-containing image of the frame in coordinates The total intensity value of the pixel. In this embodiment, in a very low-light scene with a streak camera, the photon signal can be described using a heteroscedasticity model based on photon counting (Poisson) and readout Gaussian noise; wherein, the heteroscedasticity model is specifically: ; In the formula, For the first Frame image in coordinates The number of photons reaching the pixel at that location is [number]. for The image frame number containing the signal image. For the first Frame containing signal image in coordinates The photon arrival count at a location follows the parameter: The Poisson distribution, For the first Frame containing signal image in coordinates The average photon arrival rate at the location; This is the gain / response coefficient (which can vary with pixels); coordinates A fixed background item at a given pixel; For the first Frame containing signal image in coordinates Readout noise value at the location, For the first Frame containing signal image in coordinates The readout noise term at point follows a mean of 0 and a variance of . The normal distribution coordinates The readout noise variance corresponding to the pixel at that location.

[0029] In this embodiment of the application, regarding coordinates Pixel, based on the mean of background noise Pixel noise standard deviation The total intensity value of the pixel in the signal-containing image at that location is used to perform pixel-level background noise removal and noise normalization, resulting in a normalized sequence; specifically: ; In the formula, For the first Frame containing signal image in coordinates The pixel sequence values ​​after standardization. For the first The original signal-containing image of the frame in coordinates The total intensity value of the pixel at that location. coordinates Mean background noise of the pixel coordinates The standard deviation of pixel noise at a given pixel; To prevent stable terms with a denominator of zero, .

[0030] In this way, by eliminating the influence of fixed background at the pixel level and mapping data at different pixel levels / noise levels to a comparable scale, the separability of extremely weak signals in the normalization space is improved.

[0031] Step S102: For each pixel, perform statistical feature extraction on the standardized sequence to construct a pixel-level statistical feature tensor; at the same time, logarithmically compress the pixel noise standard deviation of the pixel as a noise conditional feature, and concatenate the pixel-level statistical feature tensor and the noise conditional feature in the channel dimension to form a pixel-level joint feature vector.

[0032] In this embodiment of the application, the time dimension is statistically analyzed. Frame-containing signal image normalization sequence Four types of statistical features are constructed, and the statistical features are concatenated by channel to obtain a statistical feature tensor. In order to make the enhancement intensity adaptively adjust with the noise level, the standard deviation of pixel noise is used as a conditional input and dynamic range compression mapping is performed.

[0033] Considering the characteristics of photon signals in extremely weak light scenarios, such as sparseness, random arrival, and sporadic peaks, this embodiment statistically analyzes the time dimension. Normalized sequence of frames containing signal images (normalized series) Four types of statistical features are constructed for each pixel position. , and put it in All in the frame It is viewed as a time series, and four types of statistical features are constructed, specifically: Time mean characteristics , ; Time variance characteristics , ; Global time series maximum value characteristics , ; Effective frame count features , ,in, For indicator functions, The empirical threshold is used; a valid frame refers to... The number of frames that exceed the empirical threshold.

[0034] Thus, unlike traditional direct multi-frame integration, this embodiment extracts four types of statistical features from the image sequence from the time dimension: mean, variance, extreme values, and over-threshold counts, and concatenates them into a multi-channel feature tensor. This multi-angle representation method can more comprehensively capture the sparsity and randomness of signals under extremely low light conditions, avoid the problem of noise amplification caused by simple accumulation, and significantly improve the separability of weak signals.

[0035] In this embodiment, for each pixel location, the above four types of statistical features are concatenated by channel to obtain a pixel-level statistical feature tensor. : ,in, This indicates splicing processing along the channel dimension. Furthermore, to ensure the enhancement intensity adaptively adjusts with noise levels, the standard deviation of pixel noise for each pixel is... As a conditional input, and subjected to dynamic range compression mapping, specifically: ; in, The noise condition characteristics after compression. For logarithmic mappings (such as...) ), The standard deviation of noise. It is a stable term.

[0036] Thus, a pixel-level joint feature vector is ultimately formed as the network input: ,in, It is a pixel-level joint feature vector.

[0037] Step S103: Construct a U-Net network that incorporates residual learning. The decoder of the U-Net network is a multi-scale hierarchical structure symmetrical to the encoder. The encoder and the decoder perform same-scale feature fusion through skip connections.

[0038] In this embodiment of the application, the pixel-level joint feature vector As network input, the spatial dimensions of the input tensor are consistent with the original image, and the channel dimension consists of statistical feature channels and noise conditional channels. (Refer to...) Figure 6 The network adopts an encoder-decoder U-Net structure and introduces a residual learning mechanism to enhance the extraction of extremely weak signals and suppress noise components.

[0039] Specifically, the U-Net network in this embodiment comprises three parts: an encoder, a bottleneck layer, and a decoder. The encoder is used to extract multi-scale representation information of the input data step by step, and the decoder is used to restore spatial resolution and reconstruct the enhanced result step by step. The encoder and decoder perform same-scale feature fusion through skip connections to preserve the detailed structure of weak signals from the streak camera. The encoder is set as a multi-scale hierarchical structure with four scale levels selected, and the number of feature channels in each level increases progressively, for example, 64, 128, 256, and 512. Each level includes at least two convolutional operations, followed by a non-linear activation function. The kernel size of the convolution is preferably 3×3 with a stride of 1, and zero-padding is used to maintain the feature map size. Resolution reduction is achieved between adjacent levels through downsampling operations, which employ convolution or pooling operations with a stride of 2. The bottleneck layer is located at the end of the encoder and is used to extract high-level semantic features. It consists of multiple convolutional structures to enhance the expressive ability of weak and sparse signal patterns. The decoder is configured as a multi-scale hierarchical structure symmetrical to the encoder, recovering the spatial resolution of the feature map through progressive upsampling. The upsampling operation is implemented using transposed convolution or a combination of interpolation upsampling and convolution. In each decoding layer, features from the previous decoder are fused with the corresponding encoder layer's output features via skip connections. The fusion method is channel-dimensional concatenation, thereby achieving an effective combination of shallow detail information and deep semantic information.

[0040] In the embodiments of this application, noise condition characteristics As one of the input channels, it is input into the network along with statistical features, enabling the network to perceive the noise level corresponding to different pixel locations and adaptively adjust the enhancement strategy. In areas with high noise, the network tends to enhance noise suppression capabilities; in areas with potentially weak signals, the network tends to preserve local peaks and structural information.

[0041] This embodiment uses residual learning to output the enhancement result. The network learns the residual difference information between the input benchmark and the target enhancement result, and the network output is represented as follows: ; in, The input tensor is composed of the pixel-level joint feature vectors. For the mapping of the neural network to be trained, For the network at pixel location The residual components output at the location.

[0042] Final Enhancement Results Represented as: ; in, As a reference, the input image is located at the pixel position. The pixel value at the reference input image is selected from the time-mean feature channel or other channels that can characterize the distribution of the original signal.

[0043] In this embodiment, the network as a whole adopts a fully convolutional structure, which can adapt to different input sizes; the activation function is ReLU or a variant thereof.

[0044] Thus, by employing a multi-scale encoder-decoder architecture and using pixel-level noise standard deviation as a conditional input, the network's enhancement strength can be adaptively adjusted according to the noise level. The network focuses on the difference between the signal and noise through residual learning, effectively recovering and enhancing the structural details of weak signals submerged in noise while suppressing strong noise, thereby improving the method's generalization ability under different experimental conditions. Therefore, through multi-scale feature extraction, noise-conditional guidance, and residual learning mechanisms, effective enhancement and noise suppression of extremely weak signals are achieved.

[0045] In this embodiment of the application, since it is difficult to obtain a noiseless ground truth image that strictly corresponds to a noisy image under extremely low light conditions, a self-supervised consistency training strategy without clean ground truth is adopted to train the network, and inference is performed after training to obtain enhanced results.

[0046] During the training phase, multiple frames of background images were first acquired under conditions of no incident signal to obtain the average background noise at each pixel location. and pixel noise standard deviation This is used as a priori parameter for subsequent standardization and noise condition modeling.

[0047] Subsequently, multiple frames of raw images containing the signal were acquired under the same imaging conditions. Based on the background noise mean and pixel noise standard deviation, the original image is subjected to background correction and normalization to obtain a standardized sequence. Furthermore, statistical features are extracted in the time dimension, and combined with noise condition features to construct the network input tensor. .

[0048] To achieve unsupervised training, multiple frames of images acquired from the same set are divided into at least two subsequence groups, and input tensors are constructed for each subsequence. and Since each subsequence originates from the same physical scene, their potential effective signals are consistent. Therefore, the consistency between network outputs is used as a training constraint.

[0049] Each and Input the network and get the output results. and A consistency loss function is constructed to make the two pixels as close as possible at their corresponding pixel positions. The consistency loss function is expressed as: ; or ; For consistency loss function, This is the first output result corresponding to the first pixel-level joint feature vector. This is the second output result corresponding to the second pixel-level joint feature vector. for Norm, for Norm.

[0050] Therefore, this embodiment adopts a group consistency self-supervised training strategy, which can complete training without ideal noise-free ground truth, reducing the difficulty of data acquisition and enabling the model to learn effectively on real and complex noisy data. This reduces the difficulty of experimental acquisition and improves the usability on real extremely low light data.

[0051] In some implementations, smoothing constraints or sparsity constraints may also be introduced to further improve the quality of the enhancement results, with the corresponding total loss function being: ; in, For the total loss function, For consistency loss function, For the total variational smoothing term, For sparse constraint terms, and These are the weighting coefficients.

[0052] During training, a gradient descent-based optimization method is used to update the network parameters, with the Adam optimizer employed. The learning rate is set according to the actual task. , Alternatively, the training batch size and the number of training rounds can be set according to the sample size and convergence, but this embodiment does not limit them.

[0053] Step S104: Based on the trained U-Net network, the pixel-level joint feature vector is enhanced and reconstructed to obtain the enhanced result.

[0054] During the reasoning phase, only arbitrary input is required. The original image containing the signal to be enhanced. And call the pre-calibrated mean background noise value and pixel noise standard deviation First, regarding the aforementioned The frame images undergo background correction and normalization to obtain a normalized sequence; then, statistical features are extracted in the time dimension and combined with noise conditional features to construct the network input tensor. Finally, the input tensor Input the trained network to obtain the output augmentation result. .

[0055] In this embodiment, the inference process does not require group consistency constraints and only needs one forward propagation to output the enhanced image. Therefore, it has high processing efficiency and can be used for offline data analysis as well as online enhanced display under the condition of corresponding computing power.

[0056] Through the above training and reasoning methods, this application can achieve stable extraction and enhanced display of weak signals in images from extremely weak light stripe cameras without clean ground truth samples, thereby improving the detection sensitivity and availability of the system under low signal-to-noise ratio conditions.

[0057] This embodiment provides a method and related equipment for improving the detection sensitivity of a streak camera. It involves acquiring multiple frames of background and signal-containing images from the streak camera under the same imaging conditions. Based on the background and signal-containing images, a standardized sequence is determined for each pixel. Statistical features are extracted from the standardized sequence to construct a pixel-level statistical feature tensor. Simultaneously, the pixel noise standard deviation of the pixel is logarithmically compressed and used as a noise conditional feature. This feature is then concatenated with the pixel-level statistical feature tensor along the channel dimension to form a pixel-level joint feature vector. Finally, the pixel-level joint feature vector is enhanced and reconstructed using a U-Net network incorporating residual learning to obtain the enhanced result. Therefore, in extremely low light and low single-frame signal-to-noise ratio scenarios, the influence of fixed background noise is effectively weakened and the noise scale is unified by first performing pixel-by-pixel statistical modeling and normalization of background noise, thereby improving the separability of weak photon events. Then, the pixel-level statistical feature tensor extracted through statistical features and the pixel noise standard deviation guided by noise conditions are used as the conditional input of a multi-scale fully convolutional network that introduces residual learning, so that the enhancement intensity can be adaptively adjusted with the noise level to adapt to different gain, illumination conditions and experimental parameter settings, thereby achieving strong noise suppression and weak signal enhancement while maintaining structural details.

[0058] Example 2 To verify the effectiveness of the method for improving the detection sensitivity of the streak camera provided in Example 1, this example selected a Rhodamine B sample as the test object to verify the improvement in detection sensitivity of the streak camera under extremely low light conditions. The test focused on comparing the processing effects of the new method proposed in Example 1 and the traditional centroid method on the fluorescence attenuation signal of Rhodamine B under different time settings (9 ns and 51 ns) and different light intensity conditions (the test light intensity gradually decreased from LD15 to LD8). The gain in the test was 1000, and the exposure time was 100 ms.

[0059] Test comparison results refer to Figures 7 to 30 , Figure 7 A schematic diagram of fluorescence signal intensity at the centroid method under 9ns level and LD15 conditions provided in the embodiments of this application; Figure 8 A schematic diagram of the fluorescence signal intensity of the method of this application under the 9ns level and LD15 conditions provided for embodiments of this application; Figure 9 A schematic diagram of the fluorescence decay curve of the centroid method under 9ns setting and LD15 conditions provided in the embodiments of this application; Figure 10 A schematic diagram of the fluorescence decay curve of the method of this application under 9ns level and LD15 conditions provided for the embodiments of this application; Figure 11 A schematic diagram of fluorescence signal intensity under centroid method at 9ns level and LD11 conditions provided in the embodiments of this application; Figure 12 A schematic diagram of the fluorescence signal intensity of the method of this application under the 9ns level and LD11 conditions provided for embodiments of this application; Figure 13 A schematic diagram of the fluorescence decay curve of the centroid method under LD11 conditions at the 9ns level, provided in an embodiment of this application; Figure 14 A schematic diagram of the fluorescence decay curve of the method of this application under 9ns level and LD11 conditions provided for the embodiments of this application; Figure 15 A schematic diagram of fluorescence signal intensity at the centroid method under LD8 conditions at the 9ns level, provided in an embodiment of this application; Figure 16 A schematic diagram of the fluorescence signal intensity of the method of this application under 9ns level and LD8 conditions, provided for embodiments of this application; Figure 17 A schematic diagram of the fluorescence decay curve of the centroid method under LD8 conditions at the 9ns level, provided in the embodiments of this application; Figure 18 A schematic diagram of the fluorescence decay curve of the method of this application under 9ns level and LD8 conditions, provided for the embodiments of this application; Figure 19 A schematic diagram of fluorescence signal intensity at the centroid method under LD15 conditions at the 51ns level, provided for an embodiment of this application; Figure 20 A schematic diagram of the fluorescence signal intensity of the method of this application under the conditions of 51ns level and LD15, provided for the embodiments of this application; Figure 21A schematic diagram of the fluorescence decay curve of the centroid method under LD15 conditions at the 51ns level, provided for an embodiment of this application; Figure 22 A schematic diagram of the fluorescence decay curve of the method of this application under 51ns level and LD15 conditions provided for the embodiments of this application; Figure 23 A schematic diagram of fluorescence signal intensity at the centroid method under LD11 conditions at the 51ns level, provided in an embodiment of this application; Figure 24 A schematic diagram of the fluorescence signal intensity of the method of this application under 51ns level and LD11 conditions provided for embodiments of this application; Figure 25 A schematic diagram of the fluorescence decay curve of the centroid method under LD11 conditions at the 51ns level, provided for an embodiment of this application; Figure 26 A schematic diagram of the fluorescence decay curve of the method of this application under LD11 conditions at a 51ns level, provided for an embodiment of this application; Figure 27 A schematic diagram of fluorescence signal intensity at the 51ns level and under LD8 conditions provided in this application embodiment; Figure 28 A schematic diagram of the fluorescence signal intensity of the method of this application under LD8 conditions at a 51ns level, provided for an embodiment of this application; Figure 29 A schematic diagram of the fluorescence decay curve of the centroid method under LD8 conditions at the 51ns level, provided in an embodiment of this application; Figure 30 A schematic diagram of the fluorescence decay curve of the method of this application under LD8 conditions at a 51ns level, provided for an embodiment of this application.

[0060] like Figures 7 to 30 As shown, at two time points, 9 ns and 51 ns, as the light intensity gradually decreases from LD15 to LD8, the fluorescence signal intensity of the Rhodamine B sample gradually decreases, the effective signal in the original image becomes weaker, and the influence of background noise and random noise on the extraction of the fluorescence decay curve gradually increases. Under higher light intensity conditions, such as LD15, both the proposed method and the centroid method can extract the fluorescence signal to a certain extent, but the signal continuity and background suppression effect after processing by the new method are better. Under medium light intensity conditions, such as LD11, the centroid method begins to be affected by noise interference, and the fluorescence decay curve fluctuates more; the proposed method can still better preserve the main structure of the fluorescence decay signal, and the processing result is relatively stable. Under lower light intensity conditions, such as LD8, the centroid method's ability to extract the effective signal decreases significantly, and the fluorescence signal in some areas is easily submerged by noise; the proposed method, due to the introduction of background noise prior modeling, statistical feature extraction, and noise condition enhancement reconstruction, can further suppress noise and enhance weak fluorescence signals under weak light conditions, making the fluorescence lifetime curve easier to extract and fit.

[0061] Example 3 Based on the above embodiments, this embodiment also provides an apparatus for improving the detection sensitivity of a stripe camera, used to implement the above-described method for improving the detection sensitivity of a stripe camera. The apparatus of this embodiment includes: The sequence acquisition module is used to acquire multiple frames of background images and signal-containing images from the streak camera under the same imaging conditions, and to determine a normalized sequence for each pixel based on the background images and signal-containing images; the normalized sequence is the pixel sequence value after eliminating the influence of a fixed background for each pixel. The feature extraction module is used to extract statistical features from the standardized sequence for each pixel and construct a pixel-level statistical feature tensor. At the same time, the standard deviation of pixel noise is logarithmically compressed and used as a noise condition feature. The pixel-level statistical feature tensor and the noise condition feature are concatenated in the channel dimension to form a pixel-level joint feature vector. The residual learning module constructs a U-Net network that incorporates residual learning. The decoder of the U-Net network is a multi-scale hierarchical structure symmetrical to the encoder. The encoder and the decoder fuse features at the same scale through skip connections. The result generation module is used to enhance and reconstruct the pixel-level joint feature vector based on the trained U-Net network to obtain the enhanced result.

[0062] Figure 31 This is a schematic diagram of an electronic device provided in an embodiment of this application. The electronic device 20 specifically includes: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in a method for improving the detection sensitivity of a streak camera disclosed in any of the foregoing embodiments. Alternatively, the electronic device 20 in this embodiment can specifically be an electronic computer.

[0063] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0064] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored thereon can include operating system 221, computer program 222, etc., and the storage method can be temporary storage or permanent storage.

[0065] The operating system 221 is used to manage and control the various hardware devices on the electronic device 20 and the computer program 222, which may be Windows Server, Netware, Unix, Linux, etc. In addition to a computer program capable of performing a method for improving the detection sensitivity of a streak camera as disclosed in any of the foregoing embodiments, the computer program 222 may further include computer programs capable of performing other specific tasks.

[0066] This application also provides a computer-readable storage medium storing a computer program. When executed by a processor, the computer program implements the method in any of the embodiments of this application. Specifically, a system or apparatus equipped with a storage medium may be provided, on which software program code implementing the functions of any of the above embodiments is stored, and the computer (or CPU (Central Processing Unit) or MPU (Microprocessor Unit) of the system or apparatus may read and execute the program code stored in the storage medium.

[0067] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a computer-readable storage medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication component, and / or installed from a removable medium. When the computer program is executed by a central processing unit (CPU), it performs the functions defined in the system of this application.

[0068] It should be noted that the computer-readable storage medium shown in this invention can be a computer-readable signal medium, a computer-readable storage medium, or any combination thereof. For example, a computer-readable storage medium can be an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM) or flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this invention, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. The transmitted data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The computer-readable signal medium can also be any computer-readable storage medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable storage medium can be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF (Radio Frequency), etc., or any suitable combination thereof.

[0069] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0070] The units described in the embodiments of the present invention can be implemented in software or hardware, and the described units can also be located in a processor. The names of these units do not necessarily limit the specific unit itself.

[0071] It should be noted that although several modules or units of the device for performing actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to embodiments of the present invention, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0072] In the several embodiments provided in this application, it should be understood that the disclosed systems, modules, and methods can be implemented in other ways. For example, the module embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, or indirect coupling or communication connection between modules or units, and may be electrical, mechanical, or other forms.

[0073] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit it. This application is not limited to the exact structures described above and illustrated in the accompanying drawings, and it should not be considered that the specific implementation of this application is limited to these descriptions. For those skilled in the art, various changes and modifications made without departing from the concept of this application should be considered to fall within the protection scope of this application.

Claims

1. A method for improving the detection sensitivity of a streak camera, characterized in that, include: Multiple frames of background images and signal-containing images are acquired by a stripe camera under the same imaging conditions. A normalized sequence is determined for each pixel based on the background images and the signal-containing images. The normalized sequence is the pixel sequence value after eliminating the influence of a fixed background for each pixel. For each pixel, statistical features are extracted from the standardized sequence to construct a pixel-level statistical feature tensor; at the same time, the standard deviation of pixel noise is logarithmically compressed and used as a noise conditional feature, and the pixel-level statistical feature tensor and the noise conditional feature are concatenated in the channel dimension to form a pixel-level joint feature vector. A U-Net network incorporating residual learning is constructed. The decoder of the U-Net network is a multi-scale hierarchical structure symmetrical to the encoder. The encoder and the decoder are fused with features of the same scale through skip connections. The pixel-level joint feature vector is enhanced and reconstructed based on the trained U-Net network to obtain the enhanced result; The step of extracting statistical features from the standardized sequence and constructing a pixel-level statistical feature tensor includes: For each pixel position, the single pixel position is in All standardized sequences in a frame-containing signal image are treated as a set of time series, and time mean features, time variance features, global time series maximum features, and effective frame count features are constructed. The time mean feature, the time variance feature, the global temporal maximum value feature, and the effective frame count feature are concatenated by channel to obtain a pixel-level statistical feature tensor.

2. The method for improving the detection sensitivity of a streak camera according to claim 1, characterized in that, The method involves acquiring multiple frames of background and signal-containing images from a stripe camera under the same imaging conditions, and determining a normalized sequence for each pixel based on the background and signal-containing images, including: Data was collected under conditions of no incident signal from the streak camera. The pixel intensity sequence of the frame background image, and... Perform pixel-by-pixel statistics on the background image of the frame, and calculate the position of each pixel in the frame. Mean background noise and standard deviation of pixel noise in the frame background image; collection The pixel intensity sequence of the frame contains the signal image, and the signal intensity value for each pixel coordinate is determined. For each pixel coordinate, pixel-level background noise removal and noise normalization are performed based on the mean background noise, the standard deviation of pixel noise, and the signal strength value to obtain a normalized sequence.

3. The method for improving the detection sensitivity of a streak camera according to claim 2, characterized in that, The collection The pixel intensity sequence of the frame containing the signal image determines the signal intensity value for each pixel coordinate, including: collection Pixel intensity sequence of a frame containing a signal image , For the first The original signal-containing image in coordinates The total intensity value of the pixel; wherein, the photon signal is described by a heteroscedasticity model based on photon counting and readout Gaussian noise, and the heteroscedasticity model is specifically: ； In the formula, For the first Frame containing signal image in coordinates The number of photons reaching the pixel at that location is [number]. for The image frame number containing the signal image. For the first Frame containing signal image in coordinates The photon arrival count at a location follows the parameter: The Poisson distribution, For the first Frame containing signal image in coordinates The average photon arrival rate at the location; Gain / response coefficient; coordinates A fixed background item at a given pixel; For the first Frame containing signal image in coordinates Readout noise value at the location, , refers to the Frame containing signal image in coordinates The readout noise term at point follows a mean of 0 and a variance of . The normal distribution coordinates The readout noise variance corresponding to the pixel at that location.

4. The method for improving the detection sensitivity of a streak camera according to claim 3, characterized in that, For each pixel coordinate, pixel-level background noise removal and noise normalization are performed based on the mean background noise, the standard deviation of pixel noise, and the signal strength value to obtain a normalized sequence, including: For each pixel coordinate: ； In the formula, For the first Frame containing signal image in coordinates The pixel sequence values ​​after standardization. For the first The original signal-containing image of the frame in coordinates The total intensity value of the pixel at that location. coordinates Mean background noise of the pixel coordinates The standard deviation of pixel noise at a given pixel; To prevent stable terms with a denominator of zero, .

5. A method for improving the detection sensitivity of a streak camera according to claim 4, characterized in that, The step of using the standard deviation of pixel noise as a noise conditional feature after logarithmic compression, and concatenating the pixel-level statistical feature tensor and the noise conditional feature in the channel dimension to form a pixel-level joint feature vector includes: The standard deviation of pixel noise for each pixel As a conditional input, and subjected to dynamic range compression mapping, specifically: ； in, The noise condition characteristics after compression. For logarithmic mapping, The standard deviation of noise. It is a stable term; ； in, It is a pixel-level joint feature vector. For the pixel-level statistical feature tensor, This is a splicing process at the channel dimension.

6. A method for improving the detection sensitivity of a streak camera according to claim 1, characterized in that, The U-Net network includes an encoder, a bottleneck layer, and a decoder; The encoder is a multi-scale hierarchical structure with four scale levels selected. The number of feature channels in each level increases progressively. Each level includes at least two convolutional operations. Each convolutional operation is followed by a non-linear activation function. Zero padding is used to maintain the feature map size in the convolutional kernel size. The bottleneck layer is located at the end of the encoder and is composed of a multi-layer convolutional structure. The decoder is a multi-scale hierarchical structure symmetrical to the encoder, and recovers the spatial resolution of the feature map through progressive upsampling; wherein, the upsampling operation is implemented by transposed convolution or a combination of interpolation upsampling and convolution; The U-Net network learns the differential residual information between the baseline input image and the target enhancement result, and the network output is represented as follows: ； in, The input tensor is composed of the pixel-level joint feature vectors. For the mapping of the neural network to be trained, For the network at pixel location The residual components output at the location; Enhanced results Represented as: ； in, As a reference, the input image is located at the pixel position. Pixel value at that location, To enhance the results.

7. A method for improving the detection sensitivity of a streak camera according to claim 6, characterized in that, During the training phase, the U-Net network divides the pixel-level joint feature vector into at least two sub-sequence groups, forming a first pixel-level joint feature sub-vector and a second pixel-level joint feature sub-vector. The first and second pixel-level joint feature sub-vectors are then input into the U-Net network, and training is constrained by a constructed consistency loss function. Specifically, the consistency loss function is: ； or: ； In the formula, For consistency loss function, This is the first output result corresponding to the first pixel-level joint feature vector. This is the second output result corresponding to the second pixel-level joint feature vector. for Norm, for Norm; During training, gradient descent-based optimization methods are used to update the network parameters.

8. A method for improving the detection sensitivity of a streak camera according to claim 7, characterized in that, Also includes: When using the aforementioned consistency loss function for training constraints, smoothness constraints and sparsity constraints are introduced, specifically: ； in, For the total loss function, For consistency loss function, For the total variational smoothing term, For sparse constraint terms, and These are the weighting coefficients.

9. An apparatus for improving the detection sensitivity of a streak camera, used to implement the method for improving the detection sensitivity of a streak camera as described in any one of claims 1 to 8, characterized in that, The device includes: The sequence acquisition module is used to acquire multiple frames of background images and signal-containing images from the streak camera under the same imaging conditions, and to determine a normalized sequence for each pixel based on the background images and signal-containing images; the normalized sequence is the pixel sequence value after eliminating the influence of a fixed background for each pixel. The feature extraction module is used to extract statistical features from the standardized sequence for each pixel and construct a pixel-level statistical feature tensor. At the same time, the standard deviation of pixel noise is logarithmically compressed and used as a noise condition feature. The pixel-level statistical feature tensor and the noise condition feature are concatenated in the channel dimension to form a pixel-level joint feature vector. The residual learning module constructs a U-Net network that incorporates residual learning. The decoder of the U-Net network is a multi-scale hierarchical structure symmetrical to the encoder. The encoder and the decoder fuse features at the same scale through skip connections. The result generation module is used to enhance and reconstruct the pixel-level joint feature vector based on the trained U-Net network to obtain the enhanced result.

10. A terminal, characterized in that, The terminal includes a memory and one or more processors; the memory stores one or more programs; the programs contain instructions for performing a method for improving the detection sensitivity of a streak camera as described in any one of claims 1 to 8; the processor is used to execute the programs.

Images